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FOREWORD 



The Institute reported in this volume brought together some of 
the mosc outstanding national leaders in the field of educational 
technology. Its participants were draw from state and university 
level adult basic education personnel from among the fifty states. 

The purpose of the Institute was to givt. participants insight 
into the philosophy, po:entials, problems and techniques of programmed 
instruction and computer assisted instruction as they relate to adult 
basic education programs. 

The use of individualized instructional techniques currently 
offers a promising means to increase the efficiency of learning 
among the undereducated adult population. When appropriately 
designed, programmed instruction insures that the learner actively 
responds to information and is promptly provided feedback and 
remedial instruction within seconds after his demonstrated need 
for thpm. Computer assisted instruction affords the learner 
experiences through a multi-media approach. 

While recognizing the values to be gained in utilizing educa- 
tional technology, few adult basic education personnel understand 
how to use its techniques. During the Institute, each participant 
hao an opportunity to inspect and use a variety of PI and CAI 
materials in order to familiarize themselves with the techniques 
against the background of appropriate selection criteria. The 
infornality of instruction allowed for full discussions between 
participants and instructors. 
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The major speeches presented at the Institute were taped and 
transcribed lor publication. We are grateful to each speaker who 
edited the transcribed material and gave permission for its reproduc- 
tion in this present volume. It is hoped that the presentations 
reproduced here will convey to the reader some of the enthusiasm and 
rapport with the speakers which participants in the Institute gained 
from live presentations. 

The proceed! igs of this Institute will extend to a larger 
audience some of the benefits that accrued to those present at 
the Institute itself. 

Edga. J. Boone 

Head, Department of Adult and 
Community College Education 
North Carolina State University 

Raleigh, N. C- 
March, 1971 
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PSYCHOLOGICAL FOUNDATIONS OF INDIVIDUALIZING INSTRUCTION 



Lav/rence Stolurow, Director 
CAI Projects, Harvard Computing Center 

I want to try to give you at least one point of view with respect 
to what might be called a reor ienta ticn co learning and instruction. 

An orientation which I think is gradually becoming the point of view 
that many people are adopting and trying to put into effect. 

The need for change within educational systems, particularly 
schools, is something which hau been stressed for a number of years, 

1 think Marshall McLuhan made the statement that schools and rail- 
road people were the two societal institutions that have not been 
changed over the years. While education is the social mechanism 
by means of which we pass information on and presr lably consolidate 
it into more effective form so that individuals are able to cope with 
the future, it seems to me that yesterday's ana today's events 
indicate the tempo that is with us and suggest the need for our 
educational institutions to make some rather drastic changes. 

A number of commissioners of education have mentioned at one 
time o/ another the notion cf individualizing education. If we 
are to pick a single theme or topic the notion of individualization 
is one cf the main concepts that is being worked with in order to 
make this change v. T hich everybody feels should be coming about. There 
are a variety of bases for change. They come not only from the 
youngsters who are asking for more relevant education but also from 
adults. As you well know, there is a variety of kinds of adults. 
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I don^ think there is any adult in our society today who can 
affoid not to be concerned about education, about his education, 
simply because things are changing so vapidly that he soon becomes 
out-of-date unless he does make some effort to re-orient his think- 
ing and to develop new shills. 

First 1 would like t.e mention some tacts which 1 think identify 
some needs with respect te education. It has been estimated, for 
example, that 50 percent cf the labor force today earn their living 
in industries that did not exist when the country began. A third 
of the items on the super market shelf did not exist 10 years ago. 
One half of what we learn in school is no longer valid when ve reach 
middle age. Ninety percent of the drugs prescribed today were not 
even known 10 years ago. More mathematics, chemistry, and biology 
have been created since 1930 than in the entire prior period of 
recorded history. One half of what a grad ,ate engineer studies 
today will be obsolete in 10 years. These kinds of statistics 
indicate that we must provide not only for the education of the 
group to whom we relate, but we need to build into the institutions 
that provide that education, the means by which they can keep pace 
with the changing times. Change is obviously greatly accelerated, 
and I feel it is likely to continue to accelerate. 

The question ‘■hen is, if we are going to try to respond to 
these needs, and if we are to take the theme of i nd i vidua Hz ing 
education, generally how do we go abouL doing ill I would like to 
start by first summarising, very briefly, some concepts from the 

- 2 - 
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psychology of learning which are basic to an analysis of an inter- 
action that cakes place between a student and some source of infor- 
mation-teacher, book, or some other medium. Fundamentally, the 
psychologist assumes that man has evolved as a biological organism 
in a way which distinguishes him, in terms of his cognitive or 
rational abilities, from other animals, and these processes have 
reached a state which engineering has identified as se 1 f -organiz ing . 
This means that with higher levels of intelJectual abilities the 
individual is capable of not onJy responding to his environment, but 
also of accumulating th s experience and organizing it in ways that 
make him more adaptive. The s, 3urvival of the fittest 1 ' notion vas 
fundamental to much of our recent thinking and, unfortunately, much 
of our current thinking. 

However , with programmed instruction and more recently computer- 
aided instruction, a new set of conce* _s has been identified which 
derived fundamentally from some thinking provided us by Norbest 
Wenier, who was concerned with the human usage of human beings and 
the relationship of man to the physical environment which he himself 
has created in terms of the machine. Prior to the computer era, man 
created tools which helped him work, helped him get more labor 
accompl ished , helped him move larger masses, move them more rapidly 
and with greater force. T he computer on the other hand is <in 
intellectual or conceptlonal aid to man. It helps him in rational 
problem solving. It permits him to analyze complex problems to cope 
with problems in short periods of time which previously would have 
taken many man-years. 

- 3 - 



In order to utilize this kind of technology effectively, it 
is necessary that man relate in sone effective manner to these 
adaptive mechanisms. Now we can think not only of man adapting Co 
his environment, but of his generating machines that adapt to his 
needs. We can think in terms of instruction, of the new machines 
being adaptive machines in the sense that they can be designed Lo 
adapt Lo the indiviccal and to provide him with the optimum organiza- 
tion of information and practice which is required in order for him 
to perform more efficiently. This notion of the environment adapting 
to man as opposed to man adapting to the environment is extremely 
important thinking, and fundamental to a real appreciation of computer- 
assisted i ’ truction. Adjusting to this kind of environment is some- 
thing with which all of us will sooner or later be faced. 

Psychology itself, as a profession, lias given primary emphasis 
upon its theories of learning under conditions in which man is 
adjusting to a rather fixed, or at least specified, environment 
that maintains certain critical characteristics. Teaching in the 
classroom generally has followed this pattern where the teacher 
defines the environment for the student and those who manage to 
cope with tnat environment most effectively are considered to be 
the best students. Those who do not, are those who are dropped by 
the ways ide . 

The emphasis today on the individual io coning abou*- when we are 
developing our technology to a higher level. Consequently there is 
much confusion in thinking witli respect to the relationship of nan to 
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technology. The question is, is man the master of the technology, or 
vice versa? Related to this question is the concern which many have 
that the inclusion of equipment, sometimes referred to as hardware, 
into the classroom or in the interaction with students is a deep 
impersona 1 i zat ion of the instructional process. 

I hope that after wa have concluded this series of meetings, 
and your visits to the CAI Laboratory, you v?ill leave with at least 
a question with respect to the validity of that idea and hopefully 
you will be convinced thr.t modern hardware, in the form of the 
computer, is an adaptive device that produces a greater humanizing 
environment, a more individualized environment, than we have been 
able to accomplish in the past with individual media and unrelated 
visual aids. These have been more often a burden to the teacher than 
a help. 

Now the question is what kind of thinking is important in 
trying to cope with this situation? We have seen the advances made 
to land on the moon. I feel that some of the changes we are >eeing 
on the horizon in education are going to have an equally important 
impact upon cur whole way of thinking regarding the way in which we 
relate to one another in an instructional situation. I mentioned 
earlier that psychology had sort of laid a foundation of individualized 
instruction in the sense of one-to-one relationships with individuals 
in the learning environment but had defined those situations primarily 
in Lie i w *e of Darwin - of the survival of the fittest - namely, with 
a fixed environment and an individual vho adapts to it. In that 
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context, however, a number of very useful results and a number of 



very useful theories have emerged. While the model has changed 
somewhat, because of the conditions that are now possible, it is 
nevertheless true that some insights have been provided by psycho- 
logical research on learning theory. 

The Purpose of Research 

But before talking about some of that, research, I want to 
emphasize clearly t he fact that research on learning is not 
co-extens i^e with, and equivalent to, the research on instruction. 
Learning and instruction are quite separate processes. They 'ire 
sometimes confused and in many cases the psychology of learning is 
taught to teachers with the assumption that this is all they need 
to know in crd_r to become good teachers. When we reflect upon it, 
teaching is an active process encompassing more than just learning. 

The psychology of learning is vhe result of teaching and th»2 question 
which the teacher or the teacning system is confronted with is the 
question of how to structure the environment in which the individual 
works in order for him to modify his thinking and/or actions in ways 
which permit him to achieve certain instructional or educational goals. 
Learning Psychology 

Wc generally assume that the environment in which the individual 
finds himself becomes represented internally in some symbolic manner. 
The stimuli which exist in his environment are somehow represented in 
the nervous system, as art! his own responses. The problems of 
psychology that are nost frequently thought of with respect to learning 
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are ones in which there is an appropriate relationship established 
between these representations of stimulation and the representations 
of response. The internal process which relates stimulus represen- 
tations and response representations is frequently referred to as a 
mediating process. It mediates between the external events of 
observation and of overt behavior. To refer to this, psychologist coin 
various symbols to represent what goes cn under the skin presumably 
in the central nervous system. 

Obviously in instruction and in learning one is concerned 
not only with cognitive processes. Tn fact, in the general taxonomy 
of learning behaviors there are three groups of behaviors that relate 
to the terms that I just used; namely, the cognitive domain, the 
psychomotor domain, and the affective domain. And when we identify 
objectives for instruction, when we determine what it i r. that we 
are ultimately seeking with respect to the instruction which we 
provide students in a selected learning environment, it is with 
respect to those three domains that we have reference. 

We need to distinguish also between the associative process 
and prior level or more abstract processes upon which the very 
fundamental ones build. There are otner notions of thinking, reason- 
ing and problem solving which have varying degrees of e.ceptance and 
currency and definition but fundamentally the search of the psychol- 
ogist is one which attempts to make inferences about these processes 
which none of us can aee in any one else and frequently are unaware 



within ourselves. 



But the problem here is to build theories based upon information 
which is reliable and which permits us to infer what’s going on 
within an organism. To understand > in other words, what the psychol- 
ogist is trying to Jc is fundamental to the realization that he is 
trying to proceed in a particular manner to arrive at a stale of 
understanding about processes and to test that understanding by 
means of prediction. The idea is, one's theory is only as good as 
the predictions based upon tins theory. To the extent that the 
model for the theory h? uses allows him to make extrapolations into 
the future, then he feels that that theory is effective. Two theories 
are frequently decided upon in favor of one or the other, depending 
almost entirely on the ability of each to account for existing data 
and to predict future events. 

So this is what learning theory is about, it is concerned with 
the internalization process and the representation of it. The 
manipulation with conditions in the laboratory is more often an 
effort to understand the mechanisms that are assumed to exist within 
the individual than iu is to achieve a particular learning event 
or performance of any social value. 

However, there is a growing trend which J think is highly 
desirable, that is, an aitempt to create within the real situation 
the necessary conditions of learning, or study under those constraints 
and limitations which pci mil the learning psychologist to infer some- 
thing about the process that is involved in the performance that 
he and others have onset wd , The in 3 a is to ottemnt to create 
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labora tory-likc conditions within real environment. Now the word 
"laboratory" is considered by many to be inappropriate, or some- 
thing undesirable. Really all that it means is a set of conditions 
which permit an investigator to make some inferences that something 
is related to something eJse. 

The whole process of statistical analysis of data, the logic 
of research design, is the set of conditions which will permit 
inferences once observations are made. If one cannot draw an 
inference and relate something that happens in the future to some- 
thing existing prior to that time, then one has not moved ahead 
with respect to theory jr science and one has no more knowledge 
about what to do next time. 

Learning theory is the search to achieve ultimately a state 
of understanding which can provide guidance with respect to the 
design of instructional material, guidance with respect to 
teacher training, guidance with respect to how to teach students 
to learn to study or to work so they can learn more efficiently 
or more effectively. When I mention "mediators 11 , I am speaking 
of inferential processes for which thet e are existing experimental 
designs and conditions that permit an individual to rake an infer- 
ence that something appears to be taking place within an individual 
and that it has certain properties. We speak sometimes of represen- 
tational mediators. These are mediators, processed within the 
individual, that represent his environment. We have stimulus media- 
tors so we can speak of internalized cues of images, we have process 
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mediators which refer to the internalization of responses which 
represent action or behavior or performance in some internal way. 
Responses 

The psychologist’s use of the term ir behavior M may seem 
strange to you. I am not talking about behavior in the sense 
of the child’s good behavior or bad behavior, but any performance 
of the individual, any observable kind of action. When the 
psychologist refers to behavior, he means anything observable 
about a third party which represents his performance. 

Intentions, or goals, or motivations are critical events with 
respect to the individual’s direction of behavior, with respect to 
the selections that are made in choosing among alternatives within 
an environment. Generally it is assumed that stimuli when repre- 
sented internally are not represented in total. There is some 
fragmentary representation and this is frequently cumulative 
representation so that witn increasing experience a greater or 
more comprehensive representation of external events becomes 
internalized . 

Many of us also assume that the responses we make to a 
situation have a complex set of characteristics. Our attention i' 
U3ual 1 y focused on the most obvious characteristic of the response 
on the other hand, when a child is responding to an object that he 
can consume, there are not only the chewing and biting responses 
but also the salivary response. These 1 distinguish only to 
Illustrate the fact that some part of the total response to 



stimulation is frequently capable of appearing independently of the 
event that initiated it. The response is thus a set of responses 
rather than a single responce. Some parts of it are capable of 
independent activity or appearance and can anticipate the larger 
response c which they were originally a part. 

In addition it is frequently assumed that responses produce 
stimulation. All of you are quite aware that you can ''lose your 
eyes and lift your hand and tell the position of it. You can 
do that because you have sensory feedback from your muscles which 
tell you something about the position. Coupled with responses 
are cues, or sensory information, which give you some useful 
information bases, coupling that response with some oth ft r 
response. So responses should be thought of as cue-producing , 
as being capable of providing you with stimulation which in 
itself can be v .he c : for additional responses. It is in this 

way that we learn serial behavior, or complex motor acts, such 
as dancing or playing golf. The responses become cohesive sets 
because one becomes chained to the other. 

These are some fundamental notions that represent some of the 
current thinking with respect tc psychology of learning that give 
us somewhat of a foundation for considering certain problems in 
teaching and instruction. 

Tr ansfer of Training 

Another field of important contribution is the area of transfer 
training. Historical roots go back to about 1900 in this area. 
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It is concerned with the question of how someone learns one thing 
and its influence on the learning of something else. There are 
three kinds of results. Either the first task can facilitate 
the learning of the second, it can be neutral with respect to 
the learning of the second, or it can inhibit it. Transfer 
of training provides us with some conceptions that help us in 
understanding the larger learning unit that takes place in 
school and obviously it is our purpose in instruction and particu- 
larly in curricular development to maximize positive transfer, 
to introduce sequences of experiences which build one upon the 
other in facilitating ways as much as is possible. The under- 
standing of how this is done is a critical understanding from 
the point of view of the larger unit of learning and the larger 
units of teaching. Hence, when we talk ahcut instruction, we 
are really more concerned with transfer concepts than we 3re 
with the concepts of learning themselves. The concepts of learn- 
ing that I have outlined for you are concepts which provide the 
foundation for learning which, in turn, becomes tbn basis for 
transfer. There are many concepts of transfer training and this 
is an entire area of research and development. 

Individualized Instruction 

Since we couldn’t possibly do justice to such a broad area 
in today’s presentation, I have chosen to present some of the 
thinking about the new systems of instruction, particularly, 
computer-based systems cf instruction. I would like to share 
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with you of the thinking that has been important in guiding 

our work, which tends to put both programmed instruction and compu- 
ter-aided instruction into a more systematic framework. 

This analysis, which I have labled "the ideographic model", 
highlights the fact that we are fo;using our attention on the 
one-to-one relationship of the student to some instructional 
environment. We are not dealing with group environment, as 
such, but we are focusing on the individual. Group interactions 
are not excluded, however, and I will mention this later. 

In considering the precess> I have made a distinction among 
three sets of decisions. One set of decisions is those deci- 
sions which take place before instruction is actually initiated. 

Now it seems to me if we are serious about individualizing instruc- 
tion, then we need to begin with the individual and some of his 
characteristics. A set of these characteristics has been identi- 
fied in terms of conventional ways of representing them; the 
aptitude of the student (and this would be relevant aptitude 
for the material that is being taught); his knowledge or achieve- 
ment level prior to the entry or beginning of the instructional 
experience; and somec.ng about the personality characteristics of 
the learner which help us i.\ relating various characteristics of 
the environment to his needf and expectations. These then are 
characteristics of the student. A second set of decisions are 
based on information of importance abort the instruction, its 
objectives, the topics to be covered, the final level of performance 
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on those topics and the amount of time that we can maximally 



allow for his accomplishment of these objectives. 

Given these two kinds of information, the problem which we 
have as a live teacher, tutor, guidance counselor, or computer- 
based instructional system is the search for a teaching program. 

By program we mean two things, first of all we mean some content 
that has been defined, usually subject matter, and second, some 
method of instruction which I will call a strategy of instruction. 
We can differentiate an inductive from a deductive strategy. 

In other words we can go from specific examples to general 
principles or vice versa. For example, the same subject matter 
by providing the individual with examples of words spelled accord- 
ing to a rule, or vj could present him with the rule and then give 
him illustrative words to spell. Now one of them, the former, 
was an inductive rule or strategy of instruction; the latter was 
a deductive rule or procedure of strategy of instruction. The 
idea is that a program consists of content, a specified set of 
works, the example of spelling and the rules that are to be taught 
and, secondly, the order or sequence or manner in which this 
material is presented. We call this a set of conditions and given 
that, we have specified a variety of programs and, based upon 
experience, we can make certain basic sets of predictions. 

Cl) We can select more than one program which will achieve 
our objective with each student we're working with; then we 
evaluate the instruction we provide according to some efficiency 




criteria, such as which will do it in the minimum time, or which 
one will take the least effort, or which one makes best use of 
our existing resources. 

(2) A second alternative is tc locate a single program, 
and only one; having done this, we tnen teach, 

(3) The third alternative is one which all too frequently 
ccmes up. The idea is clearly expressed in advance, but the 
problem here is that we cannot find an appropriate match between 
these individuals and the objectives, a means by which a program 
would relate the two. Consequently we have to make some new 
decisions. We can increase the student r s information level 

and this can be done by giving hiir- some separate instruction through 
a variety of means. Another way is to change our risks and gamble 
a little more and accept some students who have a high enough 
aptitude level that suggests that possibly, through the course 
of instruction, they could compensate by some additional effort 
and increase the probability that some of them will survive, 
but, obviously, the risk is greater than with those who met the 
conditions abnve. 

(4) The o:her possibility is to change the actual objectives, 
or the time that is allowed or the final performance i?vel that 
one would find minimally acceptable for the students. 

Given a change in any of these new decisions, the informa- 
tion has to be re-pr~cessed in order to determine whether or not 
teaching will take place. These processes, I think, are real ones. 
Th,;y are not often formalized, but they need to be if a proper 
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system is to be designed that copes with the initial decision- 
making for individualizing instruction. Exactly how they are 
handled varies tremendously, aepending on the particular program, 
the group that is participating and the resources available to 
us. Nevertheless, r is set of processes has to be developed if 
we are going to begin a truly individualization of instruction. 

Now we come to the second set of decisions that has to be 
made when we are individualizing instruction. And again these 
are not unr’.que to a computer; they are not peculiar to PI or 
CAI . In my s/iew, they represent fundamental processes which 
every teacher has to go through if he or she does a complete job 
in the individualization of the instruction that is being pro- 
vided . 

We can introduce the notion of cybernetic process here. The 
word "cybernetics 11 comes from Norbest Weiner who did some very 
early work in the mathetics or modern computers. We have the 
student or learner who has to respond to something in his environ- 
ment. Now the question is, what happens when he responds. 

When the learner makes a response some standard is imme- 
diately taken into account upon which a decision is made that 
that response was correct or not - was appropriate or not, 
was acceptable or not. Nov in modern technology, programmed 
instruction and CAI, we make our performance standards implicit 
right from the start. We say that the student has to make this, 
that, or the other response before we will caH it correct, and 
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we will call these responses, these partially correct, and these 
incorrect and interpret the answers in these categories. This 
is the wh^le business of making it quite clear in advance what 
will be used in making a decision about the adequacy of the 
Ft^-ent's performance. Now the fact that you have these standards 
is not enough. One has to compare the standards with the 
response. The actual process of comparison is a separate process, 
and this becomes very apparent when one is writing computer- 
aided instruction material, or when one is developing programmed 
instruction material and has to allow the student either to make 
the comparison or have the machine make the comparison. It is 
a distinctive process sometimes fallible, or somewhat fallible, 
depending on how it is accomplished. 

When a comparison has been made, whether it is correct or 
incorrect, these things usually happen. First, some knowledge of 
results is provided the learner. He is given some information 
as to the correctness or incorrectness of that performance. 

This information is conveyed in a variety of ways, the 
simplest of which is that something else happens, the scene 
changes in some distinguishable way. In other cases, the student 
is told that he is right or wrong. Many programs of instruction 
show a* first woro in the next frame "Right” or "Wrong" to 

make it quite clear to the student that he has in fact made a correct 
or incorrect response. 
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Another thing that happens is that a rule is invoked that 
provides a selection with respect to what happens next. Tre 
simplest, least interesting, selection when you have made a 
linear program and every student has made a response to frame 
one, is that the bookie 1 , selects frame two for him. "Branching", 
for example, and "contingency management", are terms used to 
represent the fact that there are existing systems which will 
make a selection for the student based upon the response pre- 
viously made, so selection can become quit^an elaborate process. 

It is either intuitive or predetermined. Ihere are no options or a 
large number of options are set up and one of these occurs on 
some basis. In addition, some record is made and usually collected 
so that we know what response or performance went with what 
experience. That is ideal. 3ut one could simply record this; 
teachers do it all the time and they make statements to the effect 
that "Oh, that's Bill and Bill always needs to be presented some- 
thing three times before he pays attention to it", or something 
of the sort. The idea is thar. the collated record three times, or 
whatever is the recorded information, accurate or inaccurate, 
complete or Incomplete, selective or non-select ive , is available 
in the system for use and it may be utilized for making selections 
or it may not. It may simply be used in making decisions about 
grades and other uses of this sort. 

In addition, there is usually a pacer, or timer control, 
some means by which the system presents material faster or more 
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slowly to students* There i some control over the speed with 
which the interactions take place and the speed can be in the 
display, huw long it is shown, how quickly it is shown after 
response, or it can be in terms of how long the student has to 
make his response. There are a variety of controls of time, 
all of which can alter the way in which the stuuent performs 
th^ learning task. 

1 prefer to describe the instructional resources, nowever 
they are formed, as a library, and to think of the system as 
including a teacher. This would mean the selection of material 
from information already available plus items in her head, let’s 
say, but It may al«o be a computer in which data are in the memory, 
and on slides or tapes, and stored in a variety of ways. The 
selection process picks out items from such a storage unit of 
some kind, which we call a library, and puts them ia a form 
which is then presented as a display to the lesrner. The 
fact that it puts them ir. some form combined with identifica- 
tion of sor.:a content that was presented according to some 
organizational rule, gives you the program step and ultimately 
the display. Now I think those are fairly fundamental sets of 
conditions which give you a general framework with which to look 
at any interaction between the student and a ttnching system 
or teacher, whichever happens to be active at the moment. The 
idea is, then, to try to think of interaction according to the 
set of classifications in order to sort out tie functions wnlch 
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the different steps really perform. You want to keep in mind thct 
we are functionally oriented. Just what function does each 
action serve when we are interacting with students? What ir 
being accomlished by the event that is taking place - an event 
initiated either by a student or a teacher? 

Now I want to mention the third step but I won't go into 
it so completely. I like to think of this step as considering 
resource management decisions. I think that considering them 
as resource management, rather than computer-managed instructions, 
is a more useful concept. You hear more talk currently about 
CMI , that is computer-managed instructions as opposed to CAI, as 
if they are in opposition. My feeling is that if you take seriously 
the idea of a systems orientation to the instructional process, 
then putting CMI in competition with CAI is an artificial 
distinction. 

One cannot take a systems approach and ignore either one; 
the idea being that, when you identify a set of contents ^nd a 
set of rules for teaching a particular student, teaching interaction 
is taking place according to that program. While that is taking 
place, the computer is monitoring that in keeping records aid 
keeping an accounting system, so to speak, as to what Is happen- 
ing with respect to the performance of the student, with relation 
to expectations, [and] if the student’s performance comes up to 
expectations that teaching interaction continues according to 
that program. On the other hand, If the instruction ir. not 
producing the results tha^ are expected at various check points, 
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it is possible to change the program. We do this frequently, for 
example, when we call in a remedial teacher in elementary school; 
that is, we call in a specialist to assist us in teaching a student 
when we are not able to relate effectively to him. Systems also 
con do this by having 3n alternative available so that, when the 
first program selected by the preins tructional process is found 
not to work, the alternate program can be substituted. That is 
all that is meant by [this duality here and] building CAI systems 
with alternatives. Alternating or shifting from one program to 
another is only now coming into actual use; however, we need to 
study its use much move intensively. Superimposed on that kind 
of management or monitoring is the usual administrative monitoring 
which takes care of a variety of activities such as the utilization 
of the other audio and visual resources, the scheduling of teachers, 
determining when they will be available so that v:hen students com- 
plete certain objectives by one means they are grouped or sent to 
a teacher oi another computer for further instruction. By the 
management of resources we mean the ability of the system to keep 
tract of developments with respect to student progress and to 
identify the possible resources that can be utilized with the 
student w T ho has achieved a certain level of performance. 

Now we. have been talking about individualizing instruction. 

When we get to this type of problem of resource management it 
becomes appropriate, I think to be sure that w T hen we are using 
the term individualization in this context we are all thinking of 
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the same thing. By individualization in this context, I mean 
’’prescription". I do not mean learning in isolation. Many people 
translate "individualization" to mean "learning in isolation". 

Il would be quite appropriate at any point in time for "prescription" 
to contain a suggestion, or recommendation, that the individual 
work with several other individuals at that time to discuss, to 
analyze as a group, to debate, a particular issue or point which 
has been taught to that individual by some more isolated means. 

In other words, "prescription" is the better synomyn for "individ- 
ualization" than "isolation". The idea here is to prescribe the 
particular set of learning conditions, whether it is an interaction 
with other students or whether it is a debate or classroom partici- 
pation and discussion is rot critical. The critical thing is that 
the appropriate kind of interaction be identified and that arrange- 
ments be made so that it can be accomplished. It is this aspect 
of instruction which we mean by individualization. It means that 
the system is relating to the individual needs; it is not that he is 
learning in isolation. 

Well, those are the three basin sets, as 1 see it, of decision 
processes that need to be taken into account in any tot^l system 
of instruction. Whether that system is partially implemented by 
teachers or totally implemented by teachers, in other words with 
or without the support of various aids and media, is not important. 
The set of conditions, 1 think, are fundamental to any real effort 
at individualization. 
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A part of the set of decisions that needs to be made with 
respect to instruction is what might be called the "medium of 
inst ruction" , the type of instruction presented the student. You 
see here a student sitting at a teletype. This is a typewriter, 
usually thought of by teachers as something that has a negative 
value because immediately students have to learn to type. It is 
the assumption, and that is an additional burden. It is true that 
students do have to press keys* but whether they have to learn to 
type as a secretary, or at the level of proficiency of a secretary, 
is another matter. Most of the time spent in learning is time 
spent in thinking, not in mechanical action; consequently, the 
relative importance of the key board and the fingering of it is 
less in an instructional situation then in a work situation. So 
the demands of learning to use the typewriter, in terms of location 
of the appropriate keys, are demands which from the point of view 
of the learning situation, are relatively minimal, But one thing 
that you want to remember when you are dealing w T ith CAI is that 
the computer doesn't pay any attention to what the student is 
doing until he signals at the end of a series of responses that 
he wants it to pay attentic ; He hits a special key and that 
key permits the system tc look at all he has done up to that 
point, process it, and react to it; hence he is not tying up a 
computer while he is trying to find the right keys. The computer 
Is busy doing other things and not just waiting for the learner. 
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But more important educationally, now that we have talked a'jout 
Home of the often expressed negative feelings about the keyboard, the 



positive advantage is that when the learner types A, it is clearly 
an A, and when he types a C or P, it is clearly a C or b. Consequently 
the images generated on the paper and images that are clear and dis- 
tinct models. In other words, the level of precision is imposed 
upon the learner j which from the point of view of learning is extremely 
important. The idea is to think in terms of these conditions with 
respect to the learner than those of the mechanics used. 

If you ao look at the conditions with .respect to the learner, 
you will find that, in the psychological studies on typing taught 
tc students, one of the s'de effects found is generally that theii 
language skills also improved. Consequently, there seems to be 
more to the process of interaction with a typewriter than the simple 
mechanics of typing itself. This relates back to the distinctive- 
ness of the stimuli, their usefulness as rues. The response, 
chaining and sequencing that I mentioned, all tend to develop in 
a much clearer environment with the result that learning is more 
precise and more effective. 

Another kind of console which students use deals with a multi- 
media or multi-sensory environment. Here we are dealing not only 
with the mechanical keyboard and the learner’s response to it, 
but we are also dealing with a visual display and the student has 
another response device, called the light pen. If the response is 
something that he presumably already knows, then one can simplify 
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h j j means of communicating by simply letting him point, or ore 
can have him actually construct the response, depending upon which 
objective seems to be appropriate for the instruction. Thus, the 
media or response as veil as the media of stimulation can be varied. 
This is a critical factor to be considered in terms of relating 
to students on an individual basis. By individualization we are 
thinking of systems, or teaching environments, in which the partic- 
ular kind of sensory exposure of the students is dependent upon 
the educational considerations, the objectives, as well as upon an 
awareness of the particular needs of the students and their ability 
to cope with the kinds of material that are being presented. In 
this way, ve have flexibility and the learning is scheduled on a 
point-f or-point basis in small units of time. This method is 
more efficient than shewing a film for fifteen minutes or more, 
then having students go off to practice before responding. The 
practice and or responding can take place while the individual is 
viewing and on a step-by-step basis. Complete recording is main- 
tained and decisions can be exercised with respect to the logic, 
or flow, of strategy of the instruction. So in a sense the consoles 
that have been provided car be thought of as the counterparts of man’s 
eyes and ears for an instructional system. And the computer can 
be thought of as the central nervous system that controls or 
organizes or integrates these into a totality that is cohesive, 
that has continuity. I want to point out to you that when you 
speak of CAI you should not also speak of media. A CAI system it 
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noc a medium as is a film, or a slide, or audio tape, or video tape. 

It is a computer connected to a variety of devices that organizes 
and integrates them and puts them into some kind of systematic 
relationship. As a result, CAI is a system. The distinction that 
I am making now is between these audio-visual media that are 
connected up and are a part of varying system designs, and CAI the 
way in which we could, if you will, vary the information that is 
presented to students. Not only do the media make a difference 
with respect to learning but also the decoding makes a difference 
with respect to learning. Diagrams, graphs, various devices for 
presenting information to students need to be looked at not only 
in terms of their diagramatic character of visual appearance, but 
also in terms of what they seem to require the individual to do 
once he is exposed to one or the other, or is required to relate 
one thing to another. To illustrate, let’s use some data from 
study dealing with some problems too of individualization. We 
presented a set of problems to the students; we asked these students 
to solve each problem using one of four rules. Each student solved 
a set of AO problems, in which each rule was required equally often 
so that there were 10 instances that required rule A, 10 for B, 10 for 
C, and 10 for D. We looked at the data produced by the students and 
asked the question about each student; namely, which rule does the 
student have a better batting average with, which rule when it is 
appropriate is he more likely to use correctly, what’s the probability 
of his successful use of that rule, and which rule does he apply most 
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rapidly, or most effectively. We had two kinds of information 
stored as the result of this initial experiment - (1) how effi- 
ciently a student uses the rule, and (2) how rapidly he uses it. 

On the basis of these two kinds of information, it is possible 
to set up a ratio and to say that a student has a particular 
pattern of performance which might be considered his optimum 
or best performance, given a lot of problems to solve involving 
those four rules. Then we have the computer generate for each 
student a set of new problems which fit his best performance. 

So if rule A then B, and C and D was his best sequ^ c: • , in ^ther 
words, if he tended to work best with rule A, next best with B, 

C and D, we would define his strategy as that sequence A, B, C, 

D. We would then present a new set of problems in which A was in 
fact more often the appropriate rule than B and B more than C and 
C more than D. If another student on the other hand performed 
best in a D, C, B, A order, just the reverse, then the computer 
generated a new set of problems for him so that D was the most 
likely solution to the problem, C the next most likely. Each new 
set of problems for each student thus conformed to the pattern 
we had observed in his first performance, and the question we then 
asked ourselves was; To what extent do students perform according 
to their best efforts? Do they learn to perform in a reliable way 
without explicit or specific instructions? Now we have the system 
adapting to each student to provide him with the best set of chances 
for success If he behaves in the second set as he behaved in the first, 
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but without telling him that. After a series of problems and 
solutions, we asked each student what was the order in which he used 
each of the rules, and we found that the students in terms of their 
awareness of their behavior were only about 40 percent correct, that 
is they r t ld tell us the order in which they used the rule only 
about percent of the time. AcLually they deviated from their 
first performance after we had adapted to their needs as much as 
70 percent indicating that contrary to many discovery learning 
addicts in complex problem solving situations when the environment 
is made to fit their needs, students do not discover their most 
efficient strategy. Of course, we can’t generalize on the basis 
of one small study about all discovery situations but at least 
our research indicated to us that one probably needs to provide 
students with an explicit description of their own performance 
in ways which allow them to respond according to their own ability 
in the most efficient manner. Hence individualization by adapta i n 
without instruction as to the condition does not se m to be suffi 
cient for most learners. 

There are other ways in which we can individualize. There i. 
the review option. Most CAI systems allow the student to go bach 
and review slides or tapes or parts of programs. There is a 1( go 
option that tells the system to go to a particular part of slide 
tape if there are audio-visual devices. But the idea is to look 
the conditions under review as needed and to determine from the 
characteristics of the learner the optimum tine for review with l 
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notion that people probably need to be taught habits of review which 
are a part of the general stusy habit. We can demise more explicit 
ways of having the instructional material actually assist the learners 
in reviewing. We can remind them to review, for example, if the 
data were to suggest that they were at the point in learning when 
they are likely to have confusion or conflict with respect to the 
new material and the old. This is another option to teach students 
good habits of individualized self-study, so they can actually 
become self ■ teachers . In other words, our larger objective in this 
series of examples is to try to teach each student ways in which 
he can become more effective ar his own teacher. We feel that 
this needs to be made an explicit part, rather than an implicit 
part, of an instructional program. The idea in this particular 
program is to tell the student that he can at anytime take notes 
or comment on the program so that he can record information that 
he later w'ants to review. And what we want to do here is to 
identify the kinds of notes the students take, the kinds of infor- 
mation that help them, and then to see if we can improve their 
ability to take notes, to summarize their o"n statements, so that 
when they go back to reread them the students can make sense of them. 
The idea is to first provide the capability then to study the behavior 
and then from that behavior to see what one would teach students so 
they may be more efficient at this kind of self-study activity. 

Another kind of individualization of the instructional process 
can be used. There is a difference in the way in vhicn students 
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react to the same instructional experiences. We tool, the same 
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program and set up four conditions. In one condition, the material 
that the student saw after he responded was neucral, in the sense 
that it simply told him ,r Right or Wrong". In other words it gave 
him knowledge or results, but it did not evaluate his performance. 

In the second condition, students when they were wrong were told that 
[it] was a bad response, or "you tan do better than that", and 
similar kinds of evaluative statements. A set of 14 different 
comments that had been scaled by judges were used to give the feed- 
back when the responses were wrong but not when they were right . 

In still another group, the same program of instructions was used 
except that, now, when the student was right and also, to evaluate 
his performance, which meant that it said "That’s a great response", 
or 'you are doing very well", and such encouragning statements which 
also had been rated by judges as being very positive. In the fourth 
condition, the group received both positive and negative evaluations 
depending upon whether they were right or wrong, of course, so ^hat 
they got the full treatment, so to speak. 

I should mention one other thing. To all students we gave a 
personality test. You remember my pre-selection, predecicion model 
in which the students are given different programs depending upon 
their personality characteristics as w T ell as aptitude. These 
students were assigned on a random basis so that we did not know 
which student would get what one of four treatments. The question then 
vas what is the relationship between personality characteristics 
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and a student's performance on the program in terms of achievement 
or amount learned. We found that the correlation goes from as 
high as a positive 35 to as low as a negative 50 depending upon 
the group. The relationship between aggression and attxtude 
toward the program shows a variation from negative 50 to a positive 
73. Deference, another personality characteristic and attitude 
toward the program goes from essentially zero to a minus 50 
correlation. And deference and time speed with which they learn 
goes from a negative AO to as high as a positive 48. Now I don't 
want to concern you with the details about the groups because my 
main point is simply to indicate that one kind of change seems to 
have a significant effect on determining which type of student, 
in terms of personality characteristics, dees well and what the 
student's attitude is toward the learning experience. it turned 
out for example that students who were high in aggression performed 
best when only negative evaluative statements were provided, no 
positive ones, and the; 7 also like this program better. This indicates 
that a student who is high in aggression on a personality test, 
seems to prefer a learning environment that fights back, so tc speak. 
Now you can interpret it in different ways of course. Interpreta- 
tion at tills point, in my mind, was critical in that one character- 
istic of the learning experience can make such a significant 
difference in terms of student performance, 

In summary, in terms of individualization of instruction, we 
mean not isolation but proscription personalization or humanization 
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taking into account the needs of the student, in making decisions about 

C 

his instructions on the basis of those needs. Secondly, ;hat in any 
individualization there are many factors to be considered. Some of 
them relate to the way in which the system adapts to the learner 
in sort of an automatic way, and I mentioned several of those. In 
other systems there are deliberate attempts to teach the student to 
become his own tutor, to allow him to become sensitive to his own 
need, and to practice those kinds of learning to identify the char- 
acteristics of the learning environment which should automatically 
take into account the learner, and */e should also deliberately 
implement in our instructional program those procedures which teach 
the learner to become a better self-tutor. Some of the examples 
that I gave you illustrate some of the characteristics of these 
environments which seem to play an important role. 
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THE SYSTEMS APPROACH TO EDUCATIONAL PROGRAMS 



Lawrence Stolurow 
Director, CAI Projects 
Harvard Computing Center 

My objective is to explore the systems concept, to define what 
it means in a kind of operational sense, and then to indicate to 
you more specifically where this general kind of thinking takes us 
with respect to talks about instructions. I will use the word 
"system" quite a bit, and X will also refer to something that I 
mentioned to you last time hopefully so that they will be clearer 
and more useful concepts to you- 

I think it is desirable to begin by saying that the systems 
approach means something more than systematic approach. We don't 
want to use this term as if it were synonymous with systematic. It 
is systematic, co be sure, but it has as its objective the definition 
of what might be called a system. 

General systems theory is a relatively new development in applied 
mathematics which really came about during World War II out of opera 
tions research work, particularly associated with military activity.’ 
Operations research is a rather elaborate and sophisticated area in 
which there are many texts. 

We are not going to go into this approach in detail, but this is 
an ahcempt to show with mathematical models various interrelationships 
among complex systems. In addition, there was an historical route in 
biology with an effort to organize information about biological 
processes. 
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You are familiar with the terms, such as organic systems. The idea 
was that there was rather separate recognition that one had to consider, 
not just physical entities and what they are doing. But if ve can use 

the biological analogy, we are concerned here not only with the anatomy 

of the system but also its physiology; in other words, how it works. 

And in fact, if we are to identity one of the more fundamental 
notions in the whole systems approach, it is that we should look at 
complex problems in ways which lead us to better identify the functions 
that are being performed as opposed to the identification of the objects, 
or conditions, or material of which they consist. In other words, in 
the systems approach ve are looking for functions. When one designs 
an electronic system, for example, one specifics that he wants to 
amplify some sound in a systems kind of approach <o the problem, and 
then he may use tubes or solid state circuitry co do the job. 

So tit physical unit that accomplished the function is a variable. 

It can be a variety of different physical things so the objects are 
not critic ?1. They simply represent an example of the type of function 
that is to be perfbrmed. And one has to distinguish between the physi- 
cal properties and their functional properties. For example, if I ask 

you to describe a car, most of you would start enumerating a collec- 

% 

lion of objects. You would start with something like the body, the 
wheels or the seats vhich are the objects. This is very natural, but 
I am trying to distinguish that rather natural way of looking at some- 
thing that is complex from the way in which it would be looked at if 
one were doing a systems analysis, which would be instead to say what 
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a car does. It moves people or things from place to place, and it uti- 
lizes energy. It transforms energy from one kind to another kind; so 
one deals with a systems analysis in a very different kind of language 
than one deals w T ith. let's say, an analysis of objects or a descrip- 
tion of a systems analysis in terms of its physical components. For 
example, when one talks about jobs, if you talk about a job, and 
say, "Well, 1 want to hire you for a job which Mary Smith is now 
performing, 11 you ate in a sense doing the same kind or thing; namely, 
you are identifying an individual who is filling that job, but you 
are not telling the new 7 applicant the things she has to do. 

A job description, on the other hand, tells what the individual 
that fills that job has to accomplish and it is in tjrms of doing, 
accomplishing or functioning; that is critical. 

So, one of the jobs in a system analysis is to first focus on 
the physical aspects of the problem and to identify both the func- 
tioning units and the functions they perform and to do this in 
terms of their critical characteristics as opposed to their less 
critical or variable characteristics. 

Another difference from traditional approach to problem 
solving which systems approach represents, I think, is that with 
the systems approach we enlarge the problem. We don't try to reduce 
the problem. In fact, we are trying with a systems approach to 
avoid over-simplification. In other words, the systems approach is 
more orierfed toward avoiding this error than it is upon narrowing 
dowTi a problem. Kmphasis on narrowing down a problem has frequently 
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led to misiuterpretat ion of the problem, a failure to see a problem 
in perspective. Consequently, when you do a system analysis, you 
enlarge the problem; you try to look at the problem and the environ 
nent in which it occurred because you know that the external 
environment has some implications for the problem and the main 
point of attention. 1 think it is fair to say that with systems 
analysis the orientation is toward expurgation; the effort is to 
be explicit about the nature of the problem, in fact to be complete 
rather than to oversimplify. 

I think this is very different. It sets you up differently 
in the way in which you look at problems . It means you shift your 
attention to related matters as well as specifics of the problem, 
and you try to see it in terms of some total context. 

Now the result of the systems analysis typically is the 
construction of a model, or some conception or bstraction. You 
come out of the analysis with something in symbolic form that 
represents what it is you thought you saw or that you derived from 
the analysis. And that abstraction presumably represents the 
characteristics of the problem, and is generally referred to as 
a model. 

Now models vary, of course, in the way in which they are 
formulated; some are mathematical models, and in reference to 
operational research mentioned earlier one is typically dealing 
with a mathematics model. hut a computer program is just as much 
a formalization of a problem and represents an expurgation of that 
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problem, for example, as does a mathematical model. The idea then is 
not to reduce everything to mathematics but rather to reduce tc a 
formal symbolic structure which permits you to identify the critical 
features in the problem and to see their relationship. 

Generally, you also are concerned with the problem of procedure 
because what you are dealing with for the most part are dynamic 
systems, systems that take place, that perforin over a period of 
time, that consist of activities that are not all represented at 
any moment in time but for which a time series is needed in order 
to identify all of the stages of the system. Consequently, you 
are dealing with a time series, you are loncerned with procedure or 

sequence, and this is where you are interested in what happens first, 

second, and so on. 

As you know, all computers operate on the basis of a very strict 

ordering of things in time, and everything has to be done one step at 

a time. It is done very quickly of course, so it appears to be happen- 
ing simultaneously. 

So we have both models and procedures, and generally when you 
design a model f jr a system and look at its procedures you are con- 
cerned with optimizing something. Now many people use the term 
optimizing in a very loose way. But in this context it is very 
important that you specify what is being optimized. What are you 
optimizing? it’s not sufficient to say you are optimizing, for when 
you say you are optimizing, generally you are dealing with two or 
more things. You want to optimize, let r s say, learning and retention, 
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or learning, retention and transfer of the information so that the 
individual not only learns rapidly but retains better and is able to 
use the information, which means you are dealing with several thing j. 
You want to get the best possible combination, and to get the best 
possible combination means that you get the best of each one in the 
combination. You might have to settle for a bit slower learning so 
that you get better retention, let’s say, or better transfer. 

Now these things are variable, these measures that you are 
optimizing, and you want to combine them in some way so that the 
totality is your best result in a sense but is generally referred to 
as your optimum result. The best example that I can think of now 
is that you think of man as a kind of optimization of the achievements 
of certain things. He certainly cannot see better than any other 
animal. Obviously, birds can see better than he can, he can’t run 
faster, and so on — item by item man is excelled by different creatures 
but if you take the whole combination, he tends to put together a 
very good set. This in a sense represents optimization. 

Usually systems are designed to optimize some set of things to 
achieve objectives. When you generate a model from your analysis, 
you are in a position to simulate. This means you can use the model 
to accomplish tasks more quickly than without it. For example, the 
characteristics of an instructional system can be represented symbol- 
ically as a model, then programmed into the computer to process data. 
One can use the model both to process inputs and to get outputs and 
in that way determine what would happen if various conditions exist. 
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You decide whether the model seems to be useful or effective, 
whether you want to explore what you set up from the system and 
use the model to produce for you, and whether you actually move them 
into the real world and employ them in some real setting. So your 
simulation, which is, derived from your model, gives you a way of 
examining the implications of a lot of the protl^ms that, otherwise, 
would be solved very slowly. Thus, simulation is a way of further 
exploring the validity of your model and of making predictions 
about what you might do in the real situation. In this way, you 

may narrow down the possibilities and proceed in a more efficient 

A 

manner to solve real problems. 

Not all modeling produces simulation nor is developed in this 
way and goes through the steps I have described, but this is fre- 
quently the case. And, of course, this is a very efficient way to 
proceed. From such simulations one can also derive procedures 
so that one can effect new ways of accomplishing jobs which have 
not been previously accomplished 

One of the big problems in achieving the stat< of simulation 
is in specifying or getting good measures of the things that we want 
to optimize. It is one of the big hang-ups in L..u w' ole business of 
systems approach to instruction, to education in b .M eral, the data, 
the kinds of information, or scores we get. Most people are unhappy 
with test scores; on the other hand, they are objective data, and 
on - 1 can do someth^ ug with them. If they are not s ! f’itient, then 
the question is what other ki, ds of data can we ohtili so that we can 
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gradually move from a state of undefined specification of our scores 
to greater and greater specificity. 

So we say we want to optimize not only learning; we want to 

optimize the student's attitude toward the course, whether he likes 

the course. This means you generate a questionnaire that tries to 

capture his attitude. So with each one of the things that is 

identified as being appropriate conceptually for us to achieve with 

a systems analysis, we have to back off and say how we get the data 

and demonstrate that we are in fact making some changes, making some 

improvements, { 

) 

The purpose, then, of any system has to be defined in terms of 
these kinds of objective data, and they have to be as clear as we 
can get them. Systems are different in the sense thu ;hey deal 
with analysis of a variety of problems. 

We may think of systems in terms of school management. We may 
think of systems of instruction or of systems which are designed to 
produce instructional material. These would all be different kinds of 
systems. But one could conceive of a larger system in which all of 
those systems were sub-systems, and no systems notions lead you 
naturally to a kind of hierarchical structuring of the environment. 

One can proceed either upward or downward in complexity. 

In other words, beyond the fact that one is dealing with an 
interrelated set of functions that takes some defined input and 
produces some defined output, beyond that fact one cannot specify 
the system. And the complexity of systems is another variable which 

, 40 - 

4G 



is not quite m the systems concept but is permitted in by the systems 
orientation to problems. Systems analysis does not always lead to a 
unique solution to a problem, 

I will give you a set of steps to follow in solving problems. 

Many of you probably know Dewey r s steps for problem solving. If that 
is not familiar -.o you, he says if you want to solve a problem, 
here is the way to approach it. First of all you have to identify 
what the problem is; then you have to think ebout some possible 
solutions and identify variables that affect the solution; and then 
you have to translate* this into some kind of operational form. 
Obviously that set of steps does not necessarily lead you to a 
solution of a problem. You may go through the set of steps end still 
not solve the problem, but by following those steps your batting 
average In problem solution is increased. You are more likely to 
solve problems by following that set. 

Mow, there are many different solutions for the same problem, 
in most cases. If I ask you, for example, what the rule is for 
long division, most of you will give the particular set of steps 
you were taught to believe will guarantee a solution to the problem. 
But that set of steps is not the only way to solve long division 
problems. One can, for example, solve the problem just by successive 
subtraction and find out how many times a particular number can be 
subtracted from the other. Ihen you just count those, and you should 
come up with the same answer. In fact, In modern math one of the 
procedures is to teach students a variety of ways to solve problems. 
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Now there ate two basic types of systems that 1 think are use- 
fully distinguished. One is called the open system; the other is 
the closed system. In an open system one is dealing with an input* 
some processing, and an output* which is the minimal set of condi- 
tions. But the output does not affect the system you are attempting 
to analyze* it simply goes on in an open-ended fashion. From our 
point of view, what I am doing is more or less open-ended or lends 
itself to an open-ended system analysis in that I am processing 
certain inputs, then giving it to you as output. But the reactions 
that you have to what 1 am saying* at least at the moment* aren’t in 
any way influencing what I do next. This lecturing is an open- 
ended system. 

If we talk about individualized instruction, then we typically 
are concerning ourselves with a closed-loop system, a system in 
which the response of the student does something to the teacher or 
the teaching system so as to modify what happens next. The thermo- 
stat in your house is an example of the clooed-loop system. It 
tends to turn on or off the furnace or the air conditioner, depend- 
ing on the time of year, in order to maintain a desired level of 
temperature. As the temperature varies a few degrees, the system 
goes on or off. But the system tends to do things that influence 
the on and off state. 

Students turn the instructor on and off; on the other hand, 
one student turning him off doesn't mean that the others do also, 
so in the lecture situation there isn’t that kind of control. But 




- U2 - 



48 



in a responsive environment, or an interactive environment which is 
still more complicated* we are dealing with a closed-loop, or cyber- 
netic system, in which the feed-back, the output, of the system itself 
is fed back to the system and becomes part of the input instead 
of total output. 

Fundamentally, in instructional systems, there are three basic 
components; (1) there is the learner, (2) some mechanism for trans- 
formation, and (3) some information source, which might be educational 
materials for example. And these are, at least in my view, usefully 
analyzed into a set of eight components as we go down into a lower 
level. These relate to the diagram which I used in the previous 
presentation. One of the functions to be performed is the selection 
function; another, of course, is the inout function to the learner; 
third is the learner himself; fourth is the output of the learner; 
fifth is the performance standard or expectation of the teacher or 
system with respect to the learner; sixth is the comparator, the 
process by means of which this performance and standard are related 
to one another; seventh is the information storage or library; 
and eighth is the feedback, which is the output of the instructional 
component and its response to the student. 

It is important to go ahead to some ev 2 n finer distinctions 
because we are dealing, in a sense, with an onion. Ve are peeling it 
after ve look at the problem; we are trying to see it in its totality; 
ve are trying to break it out into components and put things back 
together in a way to understand their relationship better. This is 
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the process of systems analysis. It is a fairly critical exercise to 
get the orientation that we are dealing Wj.th. Most of you, I am sure, 
are used to dealing with the s tucent-teacher interaction situations 
in a much more global fashion. One has to do that, obviously. But 
if you are to understand what systems development is all about in 
instruction, ana what the wh-vle new look in curriculum development 
(i.e., CAI and PI) is all about, then I think these distinctions 
have to come into the picture. 

Let me explain a bit with respect to selection. For example, 
when we are dealing with selection, you will hear such terms as 
lin >ar programs, branching programs, and contingency management, 
which deals with things you can put in an ,r if-then n relationship, 
such as, "if this happens, then that happens" or "if he does this, 
then the system will do that". Now that is the fundamental way in 
which most of the rules of instruction can be, and ultimately should 
be, expressed so that wa can deal with them in a fairly objective 
and efficient way. 

Psychologically, we are concerned with the characteristics of 
input. Ve want to know what the individual should attend to, look 
for, be on the alert for, and want to distinguish between what’s 
critical and what's not critical, for what's critical would be the 
fuse in the situation. Sometimes the problem in discrimination is 
just getting to know what the cues are. A person learning to read 
is obviously trying to do this. The elicitor function is a function 
to which stimulus input has to provide some immediate basis for 
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responding. The response may be instrumental, it may be a response 



that enables the individual to do something, or it may be a critical 

response; but in some sense it is a stimulus which elicits some kind 

of behavior. So the elicitor function of the input, as well as the 
cue function, needs to be distinguished; one has to ask questions as 
to which function the stimulus is performing. And if it can’t be 
associated with any particular function, then maybe it should not 
be in there. If one is not sure which but is sure that it is one 

of the two, then probably one should leave it in. 

With respect to the learner it seems to me that psychologically > 
at least with the present state of the arts, we are concerned with 
aptitude, with personality, and with knowledge as the main critical 
dimensions. These are broken down, depending upon various theorists, 
in a variety oi different ways. It is largely a matter of taste 
as to what aptitude structures you think are appropriate, and knowl- 
edge the same way. Most people, like Guilford, have identified 
what they feel are appropriate ways of looking at the learner from 
a point of view of the psychological dimensions. So there is quite 
a range of choice here, and you will not find a great deal of consist- 
ency among psychologists with respect to the way in which they look to 
the learner; but, from my point of view, if we are going to individ- 
ualize and set up an efficient structural system, we have to take 
into account information about the individual's state of knowledge, 
something about his aptitudes and about his personality. 
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Now output you might look at in terms of its form, sometimes 
referred to as the quality or various qualitative characteristics 
of it, for frequently these have meaning in relation to the tasks 



to be performed ultimate 
that are not rr presented 
then one has to modify f 



ly by the individual. If you use forms 
in the final task which he has to perform, 
or ms of presentation so that he gradually 






achieves a form that is representative of the final task to be coped 
with. One is also concerned with temporal characteristics such as 
latency and duration of the output that is provided to the student, 
the speed with which he responds, the length of time he takes to 
respond. Performance standards are typically translated into some 
statements about errors and some statements about level of improve- 
ment or rate of improvement, the level to which he will achieve, or 
the rate at which he will achieve a particular level. With respect to 
comparators, we are interested in an element by element comparison 
which is the way some systems work. They look at each element of 
the student’s performance and each element of the performance . 
standard to see how they match up, and depending upon the degree of 
match they provide certain kinds of information to the student. 
Sometimes it is just the summary or an equivalence response where 
the comparator is concerned with the end product of the response 
and not the process by means of which it was generated. 

With respect to the library, this is a very active area particu- 
larly in a computer field, how one organizes the storage of information, 
for an instructional system. Do you put things together in a man-made 
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form as we do in PI, which is the way unfortunately that most CAI is 
being generated, or does one try to develop the capabilities of a 
computer system so that general kinds of forms are specified but 
not tha details? One has lists or generators which provide some 
of the contents, and the combination produces the display that 
the individual reacts to so that man does not have to generate 
every frame and every step. The form in which storage takes place 
can be within the computer and within the memory system, an elec- 
tronic form so to speak; it can also be in film and video tape or 
these can be mixed in a variety of ways for the form, the organiza- 
tion and the sampling of contents. If you teach fractions to a 
student, for example, you don't teach every fraction possible; you 
select some fraction deliberately in accordance to some plan* When 
you get through, someone can look at the fractions you use, and 
you will have limited that set. Tl t is true of any spelling set, 
of any examples of concept; there is always some selection mechanism 
operating. So the library is the result of the notions about forms, 
organizations and tlie sampling of contents. « 

Feedback generally consists of information that has a reinforcing 
function. Information does, in fact, inform in the lay sense of infor 
mation, not in the communication sense of information; and it also 
has another aspect which you might call motivational. So when you get 
feedback while you are learning, whether you were right or wrong, 
it would decrease the probability that you would do' the same thing 
again if given the chance. If you use information that expands the 
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knowledge you have about the learning situation or about the task 
and it motivates you in either a positive or negative way, each of 
these dimensions is. more or less important and is given more or 
less attention depending upon how you, as the author, or a 
programmer, sees the system is formed. 

Now when we talk about tasks we generally distinguish in an 
analysis of learning situations between learning tasks and performance 
tasks. In all tasks, we are concerned with accuracy and quality and 
with rate and quantity. These are measures which we generally derive 
from any task. But when we are dealing with a performance task 
we are mainly concerned with the stability of the individual perform- 
ance, the reliability or consistency of his performance. 

On the other hand, if we are dealing with the learning task, 
we are interested chiefly in change. For instance, one is learning 
when he has altered his behavior, when he has progressed from a given 
point to some other point. Rather than stability, we are looking 
for change in learning tasks, so one can often determine whether 
or not the individual has put the label on, whether or not ha was 
in fact defining for himself, or for the student, the performance 
task or learning task. 

Now what kind of change you look for, of course, can vary, and 
sometimes in learning tasks we are looking for changes in rate, not 
in accuracy, where in other learning tasks we are interested in 
accuracy. Which measure you look for first is also important. Your 
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program may be set up first to produce accuracy and then an improve- 
ment in rate. 



Now that concept, that sequence when applied to the typing task, 
tends to be erroneous because, if you strive for accuracy initially 
in typing, the response which the student makes are not the same 
kind of responses that he makes when he is performing at a high 
rate. To increase his rate of speed he has to shift to a different 
kind of response, a more ballistic type of movement coming from 
the wrist; whereas, when he strives for accuracy, he uses a finger 
movement. So you are really teaching him a different skill when you 
move up to high speed. One has to worry about whether or not there 
is this kinn of transition going on; and if it is going on, then 
the question is whether you aim first for accuracy or for speed. 

Many typing teachers say, "let’s let them make errors initially and 
get the right speed, then work on errors;" so we strive for speed 
before accuracy. But this is contrary, I think, to a lot of practice. 

Instructional objectives as we state them in developing a program 
is part of our synthetic process in going from an analysis of an 
instructional task to a synthesis of it. When we define instructional 
objectives, we are in fact defining both our performance standards, 
which should be specifieJ definitely, and our error standards, which 
are a part of the performance standard, and the level of improvement 
standard. That's all tied up in one set of conditions that one is 
concerned with, and, secondly, we are concerned with the specification 
of the information content. Within the performance standard, we are 
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talking about the levels; the quality of performance, the kind of 
improvement, or change as well as the status or specific response. 

The next step is frequently one of specifying the modes of 
instruction or the ways of specifying or breaking down, making clear 
or explicit the way in which the mechanism will work. To use more 
conventional terminology, we are looking at the rules of teaching 
or those that teaching will use. This is where your theory of 
instruction, your concepts, get translated into some kind of action. 

If you don’t have a theory, and this is the way you are going to 
teach, you are still committing yourself to a mode of instruction. 
Presumably, then, you are really implying a kind of theoretical concept 
so you don’t avoid the problem. 

There is a variety of modes of instruction. I like to think of 
the modes in the following way, and I make two levels of distinction. 
The teaching level is the level at which what is useful in learning 
theory is applied to the instructional process. At the first level, 
the teaching level, I distinguish six levels or modes of interaction, 
and it is with respect to these that learning principles are used 
in varying degrees. The first mode on the teaching level is drill 
and practice. In drill and practice we define a sequence of things 
that a student will do to set up a definite order, and then proceed 
to give him experiences in which that order or sequence of events 
follows consistently from problem to problem; from drill to drill, 
and from practice session to practice session. These are highly 
structured. 
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The second mode is system control. With respect to systems we 
are always concerned about the loci of control, and with drill and 
practice we are dealing with system controlled experiences, or 
teacher specified experiences, where the system is simply carrying 
out the bidding so-to-speak of the teacher. 

A third mode is problem solving. Here we are dealing with low 
structured materials; for example, we are dealing with the situation 
in which the learner knows the language of the computer, say he knows 
whatever language his computer speaks, and he has a problem. The 
learner defines his problem, writes the code, or enters the information 
to be processed as well as the rules for processing, now using the 
computer for solving a problem. This is why 1 say it is a low 
structured situation. The student at the computer has the capability 
to process anything in this language that he puts in, and as long as 
he obeys the rules of that language, the grammar will generate an 
output for him. But he does the structuring, so the system itself 
is not highly structured. It is structured enough to be an integrated 
self-structured language, but it is the problems he puts in and 
the structure he provides that are critical. Well, this is a 
learner-controlled as opposed to a system-controlled situation. The 
system does very little; the learner controls the environment 
in problem solving. 

Inquiry is another mode. In this mode, one has a question, 
addresses it to a system, and the system provides him with an 
answer. The answer may be, M I don't understand your question; can 
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you say it in a different way?” Of course, you always have to provide 
for a question the student doesn't understand. Teachers all are 
ready for it when their students ask them questions, and systems 
must have that capability as well. 

In inquiry, we are dealing with a moderately structured 
system. There is more structure than in problem solving because 
obviously you have to compile the data base that the individual is 
asking about; and one has to have procedures in the system so that 
when the student raises a question, information, that set of words 
or set of symbols, is processed so as to make the system capable 
of relating to what it has stored. It has the language that was 
involved in problem solving plus these additional features that I 
have just mentioned. 

Another mode is the collaborative control mode in which we are 
cealing with a moderately structured situation and with a divided 
control which is shared by the learner and the system. There is 
ques tion-and-answer where the student's responses are used for 
branching, for example. 

A sixth mode is gaining. Here we are dealing again with 
moderate structure. In a gaming situation vcu are mainly inter- 
ested in giving the student experience with the symbolism or represen- 
tation of real world events and building up his ability to respond 
to these; but you are not imposing within the system a rigorous 
definition of the real vox Id situation, as you do in simulatijn. 

The objectives of gaming are mainly to give the Individual the 
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vocabulary and the response repertory, but they don't necessarily 
provide you with an adequate model of the real world., When you 
play Monopoly, for example, you learn about rents and about buying 
and selling real estate, but the controlling condition for the 
transaction is a die, or a little wheel you spin, a purely random 
entity. 

So the thing that determines what happens next is really a 
randojn processer, it is not a model of the real world unless you 
want to take the view that the real world is a random sequence of 
events, which I suppose some people might assume. Gaming is funda- 
mentally giving you the opportunity to interact with things like 
the terms of physical representation. The game might consist of 
some of the individuals actually picking up money, doing things 
with it, buying and selling things, carrying out various kinds of 
real world activities; yet, the control of how all this takes 
place, the flow of it, is not through some specific model, design, 
or any plan but is based on some kind of random generator. 

In simulation, on the other hand, the controlling factor which 
determines the fi w of events is something which has intelligence. 

It may not be a good representat ion of the real world, but it is 
the best representation you can get hold of, or work with, or 
express at the present time, so you try to simulate as you do in 
training a pilot. You try to give him the dials and representation 
of what’s going to happen when he is coming into a landing field, or 
taking off, and the system can represent the real woild quite 
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accurately. You can build in the barometric pressure and the latest 



wind conditions and get fairly good control over it. 

In other simulations like economic games where you get into 
control of economic systems as large as a country, we obviously have 
less knowledge about how these systems really work. But there are 
formal models that describe fairly well the. kinds of things that 
happen, and to the extent that they are there in determining the 
flow of events, you are dealing with a simulation. 

Of course, this is where the structure comes in, and the 
collaborative control comes in. From the fact that in a simulation 
the individual has some choice about what he wants to do and, given 
that he is committed or that he commits himself to those choices, 
the system acts accordingly. The system will do different things 
depending upon what the student does. So if you specify that he 
must take off under certain conditions, he will get certain kinds of 
experiences that are different from specification of landing; or, 
if you are talking about international banking in an economic simula- 
tion, he w T ill get different kinds of experience from that In dealing 
with household finance and budgsting of a family entity. Or, if 
you are dealing with simulations of a biological system for medical 
purposes, you will get very different kinds of interactions than if 
you are dealing with biological simulations that represent let's say 
a pharmacological or biochemical interaction. 

Those represent what seem to me to be useful distinctions among 
varieties of modes of interaction, or ways in which the mechanism I 
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referred to earlier is defined; and these are ways in which for 
instructional reasons we try to relate to students and try to pro- 
vide them with some change, some learning experience, some basis for 
altering their performance and responding at another level. 
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A. Introduction 

1. Philosophy and Rationale for Using P.I. With Undereducated 
and/or Disadv antaged Adul ts 

a. Goldberg states that the disadvantaged learner has little 
preparation for recognizing the importance of schooling in his life, 
and that early difficulty in mastering the basic intellectual skills 
which the schools and thus the broiler society demands leads to 
defeat and failure, a developing negative self-image. While it is 
not universally true that the undereducated and/or disadvantaged 
adult has not seen the importance of education for his own life 
and that of his offspring, it is frequently the case that such 
persons do not feel that they can make it in the world. The fact 
that someone knows the education is essential t.o his ovai and 
others' w T ell-being is no guarantee that the motivation and confi- 
dence necessary to enter an educational program to remain in it 
and to succeed, will be present. 

Programmed instruction is intended to generate more efficient 
and effective learning by breaking information down into a logical 
s tep-by-sLep progression, by providing immediate knowledge of 
results, by moving from simple to more complex material only after 
the responses of the student indicate that he is ready to move on, 
and so forth. 
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The individualization of instruction and the self-pacing 
features of programmed instruction have proven to be highly effective 
in enabling students to master a large amount of information and 
skills in a relatively short time span. The successes reported 
with the use of P.I. for teaching children ate many, and will not 
be discussed here. Programmed instruction, however, is particularly 
well-suited for use x^ith adults, whose experience in educational 
settings has been negative, incomplete, or both. The adult commonly 
has a long history of failure in educational settings, and, in 
some ways, is more difficult to educate than the child. Aside 
from lack of information and skills, the typical undereducated 
adult is 1f turned off M by education; he has experienced failure 
and punishment in learning situations before. Why should he 
expose himself to these aversive conditions again? 

Clearly some kind of motivational procedures are called 
for to demonstrate to the undereducated adult student that education 
can and does "pay off”, that there are positive aspects to learning. 
Of course, the big "pay off ir for education is rather delayed; this 
may take the form of better jobs, more income greater prestige, 
better education of children, and so on. It is necessary, but not 
always sufficient, to promise these kinds of rewards for partici- 
pating in educational programs. Additional kinds of rewards, more 
immediate than a distant job, are needed. Programmed instruction 
is w’ell-designed as a motivator, as w T ell as a teaching device. 
Students \ orking with P.I usually get a high percentage of frames 
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correct. In contrast to their often aversive earlier educational 



experiences, it is eye-opening to see the confidence resulting 
from correct answers to program frames. 

b. P.I. allows students to move at their own pace. 

Students whose knowledge base is poc£, or who have a very slow 
and deliberate learning style, will not be left behind, as is 
the case in the usual classroom situation. This does a great 
deal for one's confidence and for generating an environment 
which maximizes learning. P.I. also may function to reduce 
stressful competition among students. When using P.I., there 
id no necessity for other students to know how fast another 
student is learning, nor the degree of accuracy of performance. 

By utilizing P.I. we can avoid punishing students, either in 
front of a class or in private conference with a teacher, both 
of which may be extremely distasteful to any student, even more 
so to an undereducated student no has a history of failure. 

It is often the case that students' spelling, vocabulary, 
punctuation, and mastery of subject matters not covered by any 
particular program improve as a function of working with P.I. In 
order to perform effectively on programmed materials, students have 
to pick out key words and phrases in program frames. Careful 
reading is required. Thus it is not only the precise subject matter 
dealt with by a particular program that is added to the students' 
repertoires. 
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2 . Adapting P.I. for Use With Unde re ducated Adults 

In order to structure an educational environment which 
fosters learning, it is also necessary to insure that the vehicles 
of instruction, namely the learning materials and procedures 
utilized, are tailored to the student population. There is more 
to individualizing instruction than allowing students to work at 
their achievement levels, and so on. Because the undereducated 
often are members of sub-cultures which are "disadvantaged", it 
is necessary to take into account their cultural backgrounds, in 
order to develop skills and knowledge and to motivate students to 
strive for further education. 

As an example, the names, activities, settings, events, 
and phenomena referred to in programmed materials (all learning 
materials for that matter) should be those with which the students 
will be familiar. Therefore, if students come from Hispanic 
culture, such as Mexican-Americans and Puerto Ricans, concepts 
and events should be related to their backgrounds and experiences. 
One should also not lump different sub-cul tvral enclaves together 
and treat; them as if they were similar. This means that educational 
programs aimed at Pueblo Indians, such as the Hqpi, would differ in 
many ways from programs aimed at another tribe, such as the Navajo. 
Each group, the Hopi , and the Navajo, has a different cultural 
background, has different world views, has experienced different 
phenomena, and so forth. 
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We can attack local problems such as sanitation and housing 



by gathering baseline information and then including frames related 
to these problems in programmed materials designed specifically 
for a particular sub-cultural group. It is possible to combine 
the teaching of, say, English grammar, with instruction concerning 
the alleviation of bad sanitation. This is not to say that frames 
relating to a community problem will be sufficient to eradicate the 
problem. It is simply to say that by using topical information 
in program frames, we can attack more than a single concern at a 
time. This is but a single possibility for imaginative and useful 
adaptation of learning materials to the student population. 

Because the various sub-cultures within the United States 
have prohibitions and tabus unique to them, a determination of 
these should be made prior to the structuring of an educational 
program. Students should not be asked to emit behaviors which 
are incompat ible with rigid social observances in their sub-culture. 

To unknowingly include material of this type could prove harmful 
to the overall educational program, to the s tudent- teacher relation- 
ship, and to the student himself. 

It is probable that a number of you will be in charge of or 
involved in educational efforts aimed at students from differing 
cultural backgrounds. It is maybe the case thct learning materials 
which arc adapted to particular sub-cultural backgrounds are not 
commercially available. It may also be possible to redo commercially- 
prepared programs arid to tailor then toward particular sub-cultural 
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enclaves . 



In any event, the importance of such efforts for success- 
ful education cannot be stressed too strongly. The experience 
of the country of Chile provides an example of the way in which 
standard attempts to educate may have to be abandoned in favor 
of efforts which take the cultuve of the students into account. 
Chile wished to instruct pregnant women in prenatal care. They 
found that the women equated education with childishness. Since 
they didn't see themselves as children, they refused to be educated 
Solution: Chilean people placed great prestige on social clubs, 

since these were associated with the upper middle and upper classes 
Therefore, the Chilean government, through public health centers, 
arranged to have classes held in private homes, and provided te3 
and cakes. In this club-like atmosphere women began to attend 
classes ! 

If black students want only to study "Black History," 
this in itself should be no barrier to moving the focus to other 
areas of concern. By buiiding upon students' interest with 
Black History it is possible to teach numerous facts and disci- 
plines using this as a base. Science, mathematics, social studies 
and other topics can be taught in this manner, sinve all are part 
of Black History and therefore legitimate matters of concern. What 
is needed is appreciation and knowledge of cultural variables which 
may affect learning. 
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B. Relevant Research Done by Author 

A five-year research project sponsored by the Army Medical 
Research and Development Command, and undertaken at Guadalup®, 
Arizona from July 1, 1963 to June 30, 196S, used operant condition- 
ing procedures in community development activities. The town of 
Guadalupe is composed of Yaqui Indian and Mexican-American resi- 
dents. The purpose of the project vas to develop new methods for 
analyzing and influencing the behaviors of persons living in small 
social units. We had to determine environmental variables affecting 
individual and group behavior, and one? these variables were identi- 
fied and classified , develop a behavioral repertoire in the local 
population which would facilitate community development. 

Developing the population’s confidence that they could be 
instrumental in improving their community was a key problem. As 
a result of a long history of failure in their relationships with 
the world-at-large, the people exhibited a marked lack of self- 
confidence, which greatly lowered the probability of their achiev- 
ing their goals. For example, the people said that since they knew 
so little about the English language, and other important kinds 
of information, that they would never ,r get anywhere" . It was 
necessary, therefore, to devise strategies which would, first, 
build confidence and, second, develop the behavioral repertoire 
(such as facility with the English language) necessary for a 
successful community development eftort. 
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The project operated on several fronts simultaneously. One 
aspect dealt with gathering baseline information regarding social 
organization, political structure, and economic and religious 
activities, and then using this information in shaping community 
development. A second aspect dealt with an educational program 
carried out under controlled circumstances. The educational 
program began approximately one year after the starting date of 
the project or following the gathering of baseline information, 
and continued for approximately two and one-*half years. 

The population asked that English training be given high 
priority, so I designed a programmed text on English grammar that 
was specifically tailored to the students' background where feasible, 
Spanish names, e.g,, Juan or Maria, were used in place of ^nglo names. 
The situations described were those with which the students were 
familiar, and descriptions of the physical environment and objects 
(such as cars) were in keeping with the experiences of the students. 
Finally, care was taken that material contained in the frames did 
not violate the mores, norms or ethic of the '-tudents. Knowledge 
of the anthropol igical aspects of the population of Guadalupe was 
required in order to effectively tailor the learning materials 
to the students. 

Graduate students in the departments of anthropology and 
psychology were used to staff the program. Class: size was limited 
to four or five students, in order to facilitate face-to-face 
interaction of student and teacher and to allow more time for 
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analysis of student responses. Prior to beginning work with students 
the staff of three teachers received training in the anthropology 
of the town of Guadalupe. 

Seven students, ranging in age from 17 to 40 years, partici- 
pated in the educational program. Examination of their scores on 
standard tests, such as the Stanford and Metropolitan Achievement 
Tests, and the Mel nn-lJenny Reading Test, showed that the students 
possessed, on the average, a fifth grade English repertoire. The 
average student had completed about fi.'e and one-half years of school. 

Particularly in the first classes of an educational program, 
each class should contain at least one "very good” student. This 
peruon serves to demonstrate to the other students, who might he 
skeptical of their learning ability or the necessity for education, 
that persons such as they can learn and that there is a payoff for 
their efforts. The performance of the "very good” students helps 
prevent discouragement among the other students. 

Standard teaching methods were deemed inapplicable because 
our students wore universally ”drop outs”. One factor in their 
leaving school vac exposure to educational products which were not 
tailored to their backgrounds or experiences. Our students told 
us that they had a great deal of trouble with tasks assigned in 
grammar school because the terms and concepts described in these 
books held no meaning for them. 

The content of their education and the manner in which they 
were treated by their grammar school teachers led to the development 
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of avoidance behaviors on the part of the Guadalupe students. One 



approach to decreasing the aversiveness associated with learning 
situations was a series of tours around the Arizona State University 
(at Tempe), where the. research project was headquartered. Students 
vis: : ted chemistry laboratories, zoological displays, and so on. 

On t.iese occasions professors gave informal talks to the students. 

VJe often took students to campus coffee shops for get-togethers 
after class. Our students mingled with University students, 
dispelling some of the mysteries of University life and studying. 

Seme of the younger students began to inquire about the possibility 
of being admitted to college and whac would De expected of them if 
they were admitted. Verbal behavior such as " T . could like to go 
to college and learn to be somebody” increased. Attending classes 
at the University functioned as a very effective reinforcer; there 
was high prestige in k able to say "I study nights at the Univer- 

sity". 

The English program draft devised for the Guadalupe 
students served as a very effective confidence builder and teach- 
ing device, I constructed the program in two levels. The first 
level, containing 1500 frames, dealt with basic English grammar. 

The second level, containing 500 frames, dealt with reading 
comprehension. 

Students initially were quite reluctant to answer ques- 
tions in our classroom. The basis f r this characteristic of the 
students may be found in both their culture and as a result of 
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their earlier experiences in school. Deference to "superiors 11 
is a common trait among disadvantaged persons in Guadalupe. 

The verbal abuse meted out in elementary school for failure to 
respond properly reinforced this reticence to speak out. 

ihe following procedure was used to shape question answer- 
ing: before asking questions in the classroom setting, teachers 

noted the general and, where possible, specific knowledge Ivels 
of each student. As a function of interaction with students in 
their home town of Guadalupe and of chatting with them over 
coffee at campus snack bars, teachers were able to acquire a 
fairly extensive amount of information oi this sort. When a 
teacher first called on a particular srident to answer a question 
in the classroom (as opposed to uie ext ra-classroo; ) setting, he 
asr.ed a low-level question. By successively increasing the 
difficulty level of the questions asked, concirgent upon the stu- 
dent’s demonstration of ability to answer preceding questions, 
tho question- answering behavior of tne students become far more 
competent. Heavy verbal reinforcement, in the form of teacher 
statements such as "very good 11 , was contingent upon students 1 
answers. if the student was wrong positive reirf ore ement was 
vi thheld . 

If students were unable t- answer questions, this could lead 
to rapid loss of these veaki) • developed behaviors. Therefore, 
the first questi-ns were those that were determined to be answer- 
able by each individual student. At first numerous prompts were 
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supplied, such as rephrasing the question to as to provide slightly 
more information to the student. Later, prompts were gradually 
removed so that students were answering r *estions independent of 
the teacher. After ques tion-arswer ing was established, it became 
possible to ask diverse kinds of questions. While students would 
sometimes be unable to answer questions, the ehavior had been 
shaped so that they continued to answer questions. This contrasted 
with the students 1 experiences in their elementary school days, in 
which punishment frequently followed the emission of "silly" or 
"stupid" answers. 

1. Use of Reinforcers to Shape and Main tain bahavior 

rh: receipt of paperback dictionaries, grammar and reference 
books, access to "beer busts", etc., was contingent upon class 
attendance and performance. In addition to the intrinsic reinforce- 
ment of correct responding in the progtam, these extrinsic factors 
were also effective as reinf orcers . From initial baseline attendance 
levels which were irregular (approximately 50%), attendance became 
extremely regular (approaching 100%) following introduction of a 
contingency system. The students, many of whom had worked 12 
hours at a physically taxing job during the day, r. rely hissed 
classes. It was a matter of pride to have perfect attendance. 

All students attended every meeting except v T hen a death in the 
family or severe illness prohibited. 

excellent performanca was reinforced by the presentation 
of a complete set of paperback books on English grammar, vocabulary, 
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and fundamentals of reading and writing English, The quality of 
work rose dramatically following initiation of this procedure. 
Performance without error occurred dozens of times after this 
point. Students reported that they were paying more attention 
to details and checking their \. T ork more closely because of the 
payoffs involved. 

2 . Results 

The mean percentage of frames cn which students responded 
correctly w T as 94.0 for tne first level of the English program, 
and 96.1 for the second level. The percentage of correct frames 
ranged from 89.5 tc 97.1 on the first level to 92.0 to 98.6 on 
the second level. Orly one student was correct less than 90.0% 
of the time on the first level. 

Mean time for completion on the first level of the program 
was 27.50 hours, or 51.5 frames per hour, while on level tw'o the 
respective figures were 8.75 hours and 50.8 frames per hours. Time 
taken to complete the first level ranged from 16.67 hours to 37.75 
hours, while the range on the second level was from 7,63 hours to 
10.23 hours. 

The students reported that during their school years they 
had rarely been correct on tests. The students 1 initial accuracy 
on the programmed materials w T as at approximately the 50% level. 
However, being correct on every other answer v T as a large improve- 
ment over their previous classroom Performance. The students 1 
first reaction was one of surprise upon making so few’ errors, 
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relative to past performance. Later the students' accuracy and 
speed increased dramatically. 

Because the program progressively increased in difficulty 
(i.e., progressively larger step-sizes were explicitly built into 
the program), it is believed that the students' increasing famil- 
iarity V7ith the mechanics of using the program led to this signifi- 
cant improvement in performance. On the first level a typical 
student required an average of 34 minutes to complete each tf the 
first five sections, but an average of only 28 minutes to complete 
each of the second five sections. The average number of errors 
on the second five sections was only one-eighth the average number 
of errors on the first five sections. 

Initial exclamations of surprise turned to pride at masLery 
of the material. For perhaps the first time our students began 
to gain confidence in their intellectual abilities. Interest ^n 
further education and, possibly, in gaining high school diplomas, 
increased greatly. They talked less of their ''inferiority 11 ar c? 
their earlier practice of avoiding things related to education 
changed to an active seeking of further education. The program 
proved to be extremely effective, not only in providing reinforce- 
ment for engaging in learning activities, but in providing informa- 
tion to the students. A post-test, administered following completion 
of the level of the English program which dealt wi :h grammar, shoved 
that the students had mastered most, if not all, of the material in 
the program. Observation of the students' speech and writing during 
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and arter the program showed that the information acquired was 
being used in everyday activities. 

3 . S h a pin g B ehaviors Whi c h Have i . i cdiate P ayof f 

We found that the older students in the classes, i.e., 
those who had ■ -en out of school for five years or more and/or 
were over 22 years of age, had better attendance and performance 
records than did younger students. This fact has implications 
for the rehabilitation of school dropouts. Traditionally, programs 
are aimed at students who are recent dropouts. Many people assume 
that these persons should be drawn back into school before they 
drift aw’ay permanently. The longer persons are out of school, the 
harder it will be, so it is felt, to get them back into the educa- 
tional setting. 

This assumption may be far from universally correct. When 
we place a dropout in an educational program, the program is usually 
no different from w T hat he dropped ouc of. Environmental stimuli 
and aversive qualities which v’ere factors in his becoming a drop- 
out are again present. There often are no more reinforcements for 
remaining in the remedial program than there were for staying in 
school in the first place. Older persons, having been out of the 
educational setting for some time, may have had an opportunity 
for the aversive nature of such settings to fade, leading to easier 
reshaping under quite differ* at environmental conditions. Younger 
persons, perhaps v ’ery recently out of school, tend to respond with 
the same escape-avoidance behaviors previously emitted. 
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The older students in our study had an opportunity to immedi- 



ately utilize knowledge gained. For example, those employed on 
construction projects put their newly-acquired knovjledge of basic 
arithmetic to practical use and secured raises in position and 
income. Students from the ages of, say, 14 to 21 years, often 
did not hold jobs or held jobs which called only tor physical 
ability, with no opportunity to put new knowledge to work on 
the job. Efforts at behavioral analysis and design for the 
culturally-disadvantaged learner should enable these persons to 
immediately use knowledge gained in educational programs. 

This brings us to a discussion of reward and punishment 
techniques. All human societies make use of these. Reward may 
take the form of praise, appreciation, elevation of status and 
income, awarding of diplomas, and so on. Punishment may take 
the form of ridicule, fines, accusation, physical pain, confine- 
ment, restriction, etc. Therefore, the use of contingency 
systems as an adjunct to P.I. is a natural outgrowth and exten- 
sion of this concept. 

Because of the background deficits possessed by the ^aqui 
and Mex ic an-Amer lean students, it was clearly insufficient tc be 
concerned merely with inculcating certain behaviors (such as 
facility with the English language). If these students were to 
acquire mastery of enough subject areas to attain a high 
school equivalency, it w T as necessary for them to learn faster 
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than Was previously the case. The procedures devised f^r rapid 
shaping took Into account the students' very low- level baseline 
behaviors . 



By making reinforcement contingent or dependent upon 
progressively higher performance levels, it is possible to shape 
behavior in the direction of greater speed and accuracy. Jf the 
criteria for reinforcement were fixed, then tnere would be no 
compensation for change in the students* performance levels, and 
no allowance for individual differences in performance. Also, 
since material varies in difficulty as the student progresses 
through the program, there is danger in adjusting criteria too 
high or too low in relation to the difficulty of the particular 
part of the program currently being worked on. 

The adjusting reinforcement schedule technique was particu- 
larly effective with the culturally disadvantaged student, aince 
such students usually bring a poorly developed knowledge base to 
the learning situation. It is a means for providing reinforcement 
to students with very poor Dackgrounds who, in the i’3ual situation, 
would not be reinforced until their performance reached some 
absolute level, e.g., M a passing mark", "80 or 90 
C. Other Relevant Research 

1. Since 1962, programmed instruction, combined with contingency 
management techniques, has been used to increase the educational 
achievement of institutionalized offenders at the Draper Correctional 
Center in Elmore, Alabama, The work of Dr. John McKee and others 
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on his staff has been extremely successful* Utilizing P.I., it has 
been possible to individualize instruction far more than would be 
possible in the usual classroom situation. The effect of P.I. aside 
from accelerating students' progress through a sequence of learning 
materials, has been to overcome the defeatist attitude of a number 
of inmates who expected failure in anything having to do with 
education. Instruction is provided from literacy training 
through college preparatory work. A representative of hr. McKee 
is addressing the Institute, so it is not necessary o go into 
further detail here. 

2. In 1965, Harold Cohen and his co-workers from the Institute 
for Behavioral Research in Silver Spring, Maryland began an educa- 
tional program which used programmed instruction and continge:icy 
management. The program aimed at an inmate population at the 
National Training School for Boys In Washington, D. C. As with 
the Draper Project, the general purposes were: 1) to improvu the 

attitudlnal behaviors of the inmates, in that they typically were 
unmotivated, lacked persistence, and did not have proper study 
habits; 2) to improve their mastery of subject matter areas, 
since many of tha inmates not only were undereducated, but were 
also unable to acquire information and skills very easily. 

The students Cohen '-orked with at the National Training 
School were reinforced by the receipt of points for high level 
performance. The points could be used to buy goods or services, 
including leisure time, soft drinks end clothing. Other 
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reinforcement was provided by the social acceptance of bp.ing correct, 
and by the opportunity to take advanced courses following successful 
completion of basic courses. 

Like McKee T s work, the techniques used at the National 
Training School were extremely effective in improving motivation and 
developing a knowledge base. The effort was so successful that the 
methodology has been expanded and implemented at the Robert F. Kennedy 
Youth Center, which recently opened at Mortantown, West Virginia. 
Directed by Roy Gerard, who was Director of the National Training 
School, some 250 offenders are involved in a contingency-managed 
environment, utilizing programmed instruction as a major instructional 
vehicle . 

4. Current work on project newgate (Pennsylvania) suggests 
that many persons who commit crimes simply lack motivation, inter- 
personal and social skills. Although many of these persons have 
the potential to finish high school and college, they usually 
drop out of school. Once convicted and sent to prison, they 
rarely take advantage of the limited educational programs made 
available within the institutional setting. 

Faculty and students of the College of Human Development 
at The Pennsylvania State University have undertaken a project 
to help correct this situation. In a program titled, n Prcject 
Newgate 1 ', they have set out to help prepare inmates for enrollment 
in a university program. 
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Project Newgate began June 1, 1969 at the State Correctional 
Institution at Rockvlew, During the first year, 50 trainees will be 
selected from the six to eight thousand inmates committed to the 
Pennsylvania Bureau of Corrections and county jails. Preference 
will be given to inmates with high potential and poor academic 
records; to the poor and those from broken homes, tu these with 
short remaining sentences; and to blacks, who make up about half 
of the prison population. 

Of the 50 inmates selecced during the first year, it is 
expec», -d that at least 10 will be advanced enough to complete 
some college level courses and be ready for a full-time, academic 
program on a university campus after their release. Three Penn 
State professors will teach basic college courses in English, 
t biological science, and speech. Once on a college campus, the 

trainees will be provided with a stipend sufficient to cover full 
expenses including fees and tuition for their initial term. 

f Another 20 students who have finished the 10th and 11th 

! 

t grades will receive intensive college preparatory instruction 

| and will take at least three college courses before being admitted 

| to college. 

i Twenty other students classified as under achievers — 

persons who have not completed 10th grade but have high potential 
for learning — will be encouraged to perform college preparatory 
work. At the end of the project period they should have high school 
equivalency and be able to meet college admission standards. 
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Trainees in the project will be motivated to achieve their 
educational goal by a system of rewards for high-level performance. 



There will be "payoffs" for more rapid and accurate mastery of 
information. The payofi will be in the form of points, which can 
be used to obtain items such as clothing, candy, trips to campus, 
and access to entertainment. Successful academic performance 
may als^ generate such rewards as self-satisfaction, feelings of 
adequacy, mastery and success — feelings which most criminalr 
and deliquents sadly lack. 

In a new educational building at Rockview, trainees will 
be taught by the latest techniques in programmed instruction. Infor- 
mation will be presented step by step by teaching machines. Each 
new frame or unit builds on the ^receding ones . Whenever new 
information or concepts are introduced, the student is tested 
on his understanding of what is presented. Depending on his 
response, he is directed to move on to more challenging material 
or to review the lesson. In this way, trainees move along at 
tneir own speed. 

Four full-time instructors will be on hand to supervise and 
give preparatory instruction in English, mathematics, science, 
and social studies. 

Pending legislation in Pennsylvania promises to permit 
involverent of selected offenders in "off-grounds" classes and 
programs. Qualified prisoners in this project will be permitted 
limited involvement in the University’s social and cultural life. 
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Tours of the campus, ana participation in drama, art and music 
may be utilized as reinforcers and motivators for the trainees. 
Students (both volunteers and interns) will be utilized 
as academic tutors, discussion leaders, and counselors. 

In some cases, senior student counselors may be selected 
to live with prisoners in an ordinary house on the prison grounds. 
Trainees will attend academic classes full-time at a University 
after they are released. 

D. Prescriptions 

Some methods required for projects using P.1, with under- 
educated adults: 

1. Need to work with pilot groups of 5 or so students at first. 
After testing-revision cycles with these ar H other students, 
if the method pans out then expands. We need the same kind 
of testing-revision cycle for the try-out of contingency- 
management or reinforcement procedures as we do for the 
programmed materials themselves. 

2. Need for baseline information on students: a) sub-cultural 

difference'; b) differences as a function of training, 
education, etc. An example of the latter is the kind of 
information secured from the Scholastic Aptitude Test, 
Metropolitan and Stanford AchievemeziL Tests, and others. 

3. Need for fine-grain data c< ectioti systems. Need to see 
the effects on speed/accuracy/retention of P.1,, and 
contingency systems. 
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4. Need for widespread application of concept of successive 
approximation. Learning in so called "simpler" societies, 
e.g., American Indian, Is far less formalized, and relies 
much more on succnsrive approximation. Example* economic 
attainment is carefully graded from simple to complex, 

so that, in a hunting culture, a child is carefully guided 
to making a capture of a very small animal with a miniature 
bow and arrow before he is guided co the next size of animal. 
Working with undereducated and/or disadvantaged adults, we 
need to use successive approximation in depth. 

5. Need for anthropologists in educational activities, in 
designing, implementing, and analyzing. Of 1,000 staff 
members of the 20 education research and development labora- 
tories in the U.S., there is only one anthropologist (to my 
knowledge), and he is leaving for a University position. 

6. Teachers of undereducated and/or disadvantaged need anthropol- 
ogical training and experience, as well as training and 
experience w T ith sub-culture of the kind they will be working 
wi th . 

E. Wrap Up 

Programmed instruction has a natural affinity to programs 
designed to educate undereducated adults. It can, as the examples 
J have cited show, be extremely effective not only in rapidly 
providing a full education, but in providing incentive or developing 
confidence and a feeling of success in learners who sorely lack this 
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commodity. When combined with a contingency of 
such os those previously described, P.I. can be 
effective . 



reinforcement system, 
that much more 
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COMPUTER TIME SHARING IN AN ADULT BASIC EDUCATION PROJECT 



Miss Sue Stephenson 
Regional Training Offices 
Head Start 

Morehead State University 

The first operation of our program at Morehead State University 
started with one terminal located at the laboratory school on :ampus. 
The main objective of our demonstration project was to try this 
type of instruction in a rural setting. It proved very effective and 
we were funded to go on with this as a full time operation during 
the summer of 1967. We operated 36 terminals through the summer, 
c; nected only with the elementary schools. The program came from 
Stanford University to our small computer which in turn operated the 
36 terminals located at University Breckinridge Training School. 
Participants in the program consisted of students in grades 1-6 
at the training school, Upward Bound, and Neighborhood Youth Corps. 

In addition we conducted a demonstration project with a few adults, 
which was the foundation for starting the cdult program in the fall 
of 1969. 

In January 1968, the program changed hands. With the changing 
of administration, our 36 terminals were placed in different schools 
in the area, so we had 29 centers located over the eastern K ntucky 
area, one terminal in each school. Of course, the main objective 
here was to see what type of results we would receive from the 
community, administrators, teachers, and pupils. 
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The main truuble that we encountered was due to the fact that 



there was only one terminal in each school and it proved to be difficult 
for a class of 35 students to utilize one terminal. This problem was 
worked ou f by taking half the class for one semester and the other 
half the next semester. 

Our largest school is the furthest away (115 miles) which added 
to the difficulties. Some other difficulties were involved in work- 
ing with the various telephone companies. U .th the programs scattered 
over a wide area any terminal trouble was a problem, for getting a 
technician to each school in a day’s time was difficult. In our 
ueacher training we tried to discuss some of these problems so the 
teacher would know T how to handle the problem and to help us in this 
respect . 

Teacher training in the eastern Kentucky area consisted of a 
number of workshops held for teachers who would be working with 
CAI programs in the fall. Our program consisted of five workshops 
involving over 100 teachers. In these workshops we tried to give 
teachers a brief orientation to the program, an introduction to 
what they would be doing and how to integrate this program into 
their classroom activities. We discussed behavioral objectives 
and the teachers were given a chance to work on a program of 
their ovn to decide exactly what they wanted in a classroom and 
how to go about getting it. Along with this we had a session where 
they could participate on the terminal themselves as if they were 
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students. This is always interesting because adults are more afraid 
of the terminal than the students are. 

We had four areas which did time sharing. We had a time sharing 
arrangement with Eastern Kentucky Educational Development Corporation, 
the administering agent for Title III projects in our region, and 
the computer program was under Title III. EKEDC had contracted with 
Stanford for ten hours per day. Of the ten hoars, eight hours were 
utilized by the elementary schools in the area leaving only the 
last two hours from L ' ' 00 to 6:00 p.m. for use by other programs. 

Four Adult Education classes were scheduled to utilize those two 
hours. Thus, the beginning of ABE Time Sharing. 

Some of the problems encountered in our operation of the ABE 
program were due to the fact that we began late. At one time we 
had all four centers with over 300 students participating. Around 
April the program started running very smoothly but the program 
which was supposed to run through June, terminated early. At the 
termination of the project, we had 100 participants. 

We administered a survey test to the participants at the 
beginning and also at the end of the program. Another professor and 
T. developed the instrument in which there were statements made by 
participants in some of our workshops, plus questions concerning 
education :.n general. The survey was designed to get the opinion of 
the students and the teachers in the area as to the suitability of 
different arithmetic methods used in the area. We were hoping to 
get a vide variety of opinions but we didn't. We concluded that 
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the difference between the attitudes was appreciable even though 
we did not have a lot of time. The fact that none of the partici- 
pants were against the program was very rewarding because we were 
really expecting them to be. On the contrary, the students and 
the teachers in the area wanted to continue with the program another 
year . 

The achievement results wore taken from only a small group 
due to many difficulties. From one center tne pre test and post 
test results were collected for 19 students. 

Different groups have been participating in our program at 
Morehead. Some learners have been remedial students; these are 
high school students who were working on a third grade levei in 
mathematics. Other programs have been the Upward Bound program 
which ran for a period of six weeks. The gain during these six 
weeks was eight months. 

A study is still in progress to determine the feasibility 
of utilizing computers in upgrading the education system of 
eastern Kentucky public schools, col]eges, universities, and 
vocational schools. This study involves such art Q ? *.s grade 
reporting and administration. It was intended that the study 
investigate the possibility of locating a computer center in the 
eastern Kentucky area, hopefully Morehead so that we would 
supply the area much more efficiently than from Stanford, As of 
now, the study has not been completed but we are eagerly awaiting the 
report; it would eliminate a lot. of problems if it results in a new 
computer . 
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EDUCATIONAL PROJECT MANAGEMENT TECHNIQUES 



Desmond L. Cook 

Director, Educational Program Management Center 
Ohio State University 

I would like to discuss with you today the concept of: project 
management. Let me start by outlining what I mean by a project. 

The first requirement is that it should have a finite time limit, 
(i.e. the project will terminate at some point in time usually 
when the objective has been accomplished.) In ?. great many 
projects this occurs when the final report has been prepared. 

The second requirement is that there is a vide variety of tasks 
and jobs that have to be done by a large number of people utilizing 
all kinds of resources. Just think of the magnitude of some of 
the things that go in developing a Computer Assisted Instruction 
program (CAI) : You have the computer people, you have the software 

people, and you have specialists. Obviously, projects vary in their 
degree of complexity, some being very simple one-man operations 
and others very extended operations. 

It is convenient to let the project be viewed as a system. 

That system must bo defined. What we are saying here is that 
the tasks which we have to do on a project are much more like each 
other than they are to the rest of the environment in which that 
project exists. One of the major tasks in project preparation is 
defining the boundaries of the project or defining the boundaries 
of the system, and isolating the project from its environment so 
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that one is able to say that this is a task I don’t have to ao in 

order to get to my goal. One 01: the reasons a lot of projects fail 

is because of an inadequate defini* ion early i n the preparation 
of the proposal. 

Projects are usually non-repe t it ive in nature. For example, we 
can think of building a school building as a project. Now one 
advantage in building school buildings is that we have done it often 
enough that we have a model for doing it. This is quite helpful, 
but in developing CAI , for example, we don't have all the models we 
need for doing this project. Basically, we won't do a project again. 

We might get additional funding to explore another idea or a related 
idea but that will be a different project. Such projects are often 
referred to as once-through. A good case of a once- through project is 

the Apollo project. But now that we have been to the moon, the goal is 

reached; from now on going to the moon will be a matter of routine, a 
continued operation. 

Organizational Placement of Projects 

In talking about project management, one of the topics which 
must be included is the way in which projects are placed into 
organizations. Since this condition has a great deal of bearing 
upon the successful operation of a particular project. In an 
organizational structure such as a state department or education or 
a university, the structure includes various levels of management, 
functioning departments, and organizational units. Projects can 
be incorporated in several ways. 
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The project director can be placed in the organization directly 
under the administrator but away from the rest of the group. In many 
cases this is a good arrangement because the project director has 
quite a bit of freedom to hire the staff he wants, to move in the 
direction he wants. He is fully responsible. But, on the other 
band, he may be off by himself with no interaction with the insti- 
tution or agency. 

Another pattern is the placement of a project in an existing 
department or unit. In this case, the unit has the capability 
of doing the work; it has all tbe personnel needed, all the 
facilities and skills. But it. may lead to empire building and 
to little or no interaction with other units. 

The most coinmon pattern which exists in a large number of 
organizations is what is called the Matrix Structure. Here the 
project is brought in and a project director is named. He is 
responsible for getting the job done, but does so by using resources, 
personnel, and facilities from many departments and organizational 
units. In this case, the project director can make maximum use 
of these people in the unit, but since he is not in a line relation- 
ship a poor authority pattern exists. He runs into things like, 
who is going to give the rewards (the project director or the 
department head;) who gives the staff member his salary or promotion, 
and related potential conflicts. 

There are many kinds of problems associated with being a 
project manager too. For example, one of the principal job* of the 
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project director is to work himseli out of a job. He is supposed to 
bring the project to completion. What often happens is that when 
he sees the project coming to an end he is tempted to write up 
another project in order to keep himself ill this position. So the 
last four or five months of one project he may spend writing a 
proposal for another one. Consequently, we need some way of 
providing a kind of job security to project directors in between 
projects when they don 1 L h£ve to get money. You can build up an 
extremely good staff of people, but when project funds run out, 
you lose all these people. 

Fioject Manager Effectivene s is 

What makes an effective project director? First, the kind of 
educational background and experiences he brings to a particular 
project. Does he have the kinds of skills and abilities that he 
should have? This requirement is particularly important when a 
project deals with a special field such as medicine. Second 
regardless of backgrounH there are certain kinds of skills and 
abilities that a person should have, mainly logical skills and 
analytical skills. You have to be able to take something apart, 
and isolate the elements, and put it back together again. We 
find many good project managers have science backgrounds or 
research backgrounds. Thira, a project manager should have some 
experience as a project director, but this is not always possible. 

One can get this experience as an assistant project director when 
he is a graduate student. We should be aware of this when we employ 
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students and give them this kind of responsibility during their train- 
ing. It doesn't do much good to put students through a series of 
courses f'n statistics and measurements with the idea that when they 
go out they will be able to run projects. We hove to start training 
them to run projects as part of their graduate programs. Finally, 
the project director will be effective only to the extent that the 
organization puts resources and support behind him in terms of 
getting jobs done. 

Nature of Management 

Many people have made studies of management over the years. 
Historically, you could say the field of management is about fifty 
years old. People who have been spending seme time at it have 
formulated certain assumptions about what managers need from an 
educational point of viev;. These are some general ideas which now 
pervade a great deal of the training of managers. 

First, they recognize that it is a profession calling for 
common training. It doesn't make much difference whether he is a 
public or a private agency, the manager, or director, needs certain 
kinds of skills. 

Secondly, the most important task is making decisions. A 
tremendous amount of effort and energy in management is devoted 
to a study of de :ision-m2k; ng processes. This is particularly 
true with regard to resource allocation. How does the Congress, 
the Office of Education, or any other agency make decisions as to 
how it is going to ellocate the money received? It is a very big 
problem . 
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Thirdly, most management specialists say that the manager needs 

some idea of quantitative methods. Certainly, one of the prime 

reasons for this is that decision-making is always done under a 

cloud of uncertainty. Many people have simply made subjective 

decisions. Basic concepts of probability theory are needed, if 

* 

only to be able to say that if we go this way we stand a better 
chance of success than if we go another way. 

Fourth, it is recognized that the manager fundamentally has 
to ij ;et people to do things. Therefore, he spends a tremendous 
amount of time learning to work with people, handling personnel 
problems, motivational problems, and the like. The largest part 
of his time is spent in this area. 

Fifth, if we have differences in managers, it is in terms 
of different levels rather than the kinds of skills that they 

\ 

| need. For example, the people at the top level of a corporation 

| like American Motors set the policies; the managers are engaged in 

what we call strategic planning. It was the top level manager 
making the strategy decision to go into manufacturing sports cars 
instead of staying with compacts. So the differences are only in 
terms of levels, which means that we don't train specialists in 
management; ve train generalists in management so they can step 
in and operate at any level in any type of situation. 

Ve are beginning to pick this up in the field of education. 

1 Some of my colleagues in the field of educational administration 

are no longer talking about training the elementary school principal, 
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the junior college administrator and the secondary school principal. 
They are saying that there are enough common skills so that these 
people could step into any level of education and manage the enter- 
prise. In our own institutions, we are making our students go over 
more and more to political science, economics, and other areas to 
get their training to be administrators. 

Functions of Management 

Historically stated, four major management functions have been 
identified. Actually they are interrelated one to the other but 
we generally refer to them separately. (1) Planning refers to 
determination of objectives and alternatives. (2) Qrganizat ion 
refers to people, resources, and equipment associated in some kind of 
structure to help carry out the plan. (3) Directin g means getting 
people to work, the reward structure, the problem of how to deal 
with creative researchers in organized environment, and similar 
personnel problems. (4) Control refers to those actions that are 
necessary to adjust the operations to the plan. This is a basic 
idea; since it is an adjustment mechanism in a sense. It is not 
punitive. It is not negative, but if we set off to go to Chicago 
we begin to digress, we had better find a way to tell us we are 
digressing and get us back on the track before we go too far. 

When we talk about projects and the work that goes Into them, 
we find that there are tvc function^ that are a concern to us: 

One wo identify as planning, and the other we identify as control. 

In reality these functions are very closely related and it is 
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hard to separate them. We talk about planning and controlling, 
since, t.he project manager 's major responsibility centers in these 
domains » 

Project Planning and Co n tro l S teps 

In looking at a project, we can identify a series of steps 
the director goes through. The first is to set a goal or end 
objective for the project. Then, given the goal, we have to 
outline the series of supportive tasks or jobs that are going to 
help one get there. Outlining the series of tasks is nothing more 
in the long run than an objectives/hierarchy which is called project 
definition. Once this definition stage or hierarchy of objectives 
has been outlined, we know the what and w hy of a project. Then, we 
go into the work flow or plan which is a process of outlining the 
sequence or order in which the jobs have to be done moving from left 
to right. 

Then, we establish a schedule in terms of getting the project 
done by a certain date. The reason we do this task is because we 
can relate costs or budget to time and, therefore, we can plan 
and control costs and time at the same time. 

These several Steps constitute what I call establishing the 
shoulds . This is what we say should be happening; these are the 
goals we should be accomplishing; this is the work sequence we should 
be following; this is the budget ve should be following. We then 
proceed to enter control phase. Here we get periodic progress 
reports of what's going on in the project. 
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These reports tell us 



what is actually happening. Now, the project manager's job is to 
take what should be happening and compare it to what is actually 
happening, and tc identify where there are deviations. Where he 
finds significant end decisions that are necessary to get the 
project condition to where it should be. To do this, he may exer- 
cise the option of even changing the should . 

There are, however, not many choices he can make. Many of the 
discrepancies will be negative in the sense that the project might 
run behind schedule, or it might be running behind on performance. 

On the other hand, the project could be running ahead or have what 
we call a positive deviation. Things couid be going better than 
they should. 

This is what is meant by project planning and control. In 
carrying out these functions, the manager is constantly making 
decisions regarding the goals that are being achieved, the plan 
that is going to be used, the schedule that is going to be main- 
tained, and what measures of performance are going to be secured. 
Information, Decisions, and Project Management 

We know many things about decision making. We know for 
example that it involves making choices among alternatives. We 
know that ve must specify the objective of the decision before 
choosing the alternatives. We know that information is needed 
in order to make decisions. The project manager needs information. 
One of his major problems, of couise, is to determine the k_nd of 
information the manager needs in order to make decisions effectively. 
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The question is, "What kind of information does the manager need in 
a project situation?" Most people have generally identified the 
three major categories of (1) time, (2) cost, and (3) performance 
as the critical information he needed. Information about time should 
occur in the planning stage. He can then look at the schedule in 
terms of where he actually is. Is he running behind or ahead? 

Cost is preparing the budget and checking expenditures against 
this plan to make sure he is not over-spending, or under-cpending , 
or spending too fast, or too slow. By performance we mean quality 
control, making sure the objectives are being achieved, and that 
he sets out to accomplish what he said he was going to do. It 
is the qualitative dimension, if you will, as opposed to time- 
and cost-dimensions. 

Someplace or other there is a nice place to balance out these 
three dimensions. It is very interesting to think that at any 
given ;ime a project manager has to deal with time, cost, and 
performance concurrently. 

Let’s say that I am in a project situation and a certain 
problem comes up. If I want to maintain my schedule or keep time 
constant, then I can begin to play with cost and performance. On 
the other hand, if I want to maintain performance the only vari- 
ables I can play with are cost and time. If I have a budget con- 
straint, that is fixed price contract, the only things I can play 
with are time and performance. New you begin to put a constraint 
on any two of these and you begin to get into a reax problem. This 
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might occur when a schedule date is to be maintained and a budget to 
be set, the thing you begin to play with is performance. Think back 
to the Apollo program. They chose in a rather deliberate sense 
to constrain performance. They wanted a system which would guarantee 
a safe return almost 99.99 percent. They weren't too much worried 
about the cost or time. They would sacrifice a delay in getting to 
the moon in 1970 rather than sacrifice a performance. 

P roject Management Systems 

Historically, there have been many systems developed which 
helped us get the project management job done. 1 want to review 
these quickly for you. 

Let's assume for the moment that we have a little effort 
going with several tasks that have to be done. One of the earliest 
systems developed for this situation is the Gantt Chart developed 
in World War I. It outlines simply the tasks from start to comple- 
tion over a time scale. Time was used because it can be related 
to other things such as cost. While you can show the start and 
completion unfortunately one can't show the interrelation between 
Task A and Task C. This may be a very important relationship. 

The other difficulty with it is that we recognize we might be 
through a task 50 percent in time but not 50 percent in work. 

This set up now often happens, so some people wanted to 
improve on the Gantt Chart developed what is referred to as 
milestone charts. You identify within the tasks certain signifi- 
cant points of accomplishments and refer to these as milestones. 
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This enables us to nark off particular points of accomplishments so 
we can see whether or not we are proceeding on schedule. We are st 
not able, however, to see the interrelationships of one task to 
another. Gantt and Milestone systems were developed for production 
operation situations - manufacturing of cars, turning out nuts and 
bolts, and the like. 

Starting in the 1940 r s, a new phenomena was occurring. We 
began to see the development of the programmatic effort to reach a 
particular goal. The goal set out was to develop a nuclear device. 
People, agencies, resources, from all over the country were brought 
together to achieve that goal. What happened in the late 1950's 
is the development of a new management approach caused by the need 
for better project control. This led to a new variety of systems 
which are not known as network techniques. In this system, we 
have our milestone events, but also, are chute show the inter- 
relationship. Program Evaluation and Review Technique (PERT) 
is probably the classic example of this type. PERT was particu- 
larly developed for use in the research and development situation. 
Certain dimensions of it take r.are of the problems of uncertainty, 
but one has to be a student of the system to talk about it. 

Projects in Education 

What we have been trying to do is to take a lot of this kind 
of thinking and put it together for use in the field of education. 
We are saying the project is a system. A project proposal is a 
simulation or model of the system. You are trying in the project 



proposal to outline the future. You are modeling it, you are 
simulating it; you are saying, "Give me the money, and this is 
the world that I am going to create out there." This is what 
that project will look like. Of course, the better you can do 
that, the more successful the project is going to be. The less 
you are able to do it. the less you are able to model that future. 

In a sense the better you can do it, the fewer problems you will 
have . 

Let's apply systems thinking to a project as shown on Figure 1. 
In this particular example, the objective in a project is very close 
to what we call the mission function of a system. In the project 
planning we have what we call definition where we outline or 
break out terms, the function, the product, the sub-system. This 
is very similar to the factoring or decomposing that goes on in 
systems. Once you decompose a system, you put it back together 
again in terms of a synthesis and this becomes a plan. Notice in 
systems analysis column there is not scheduling dimension but there 
are feedback loop interactions. I have generated what I call a 
general project management model as exhibited in Figure 2. 

There is a basic overall system or totality. This totality 
can be broken out into two major ^ub-cat egories , one called the 
planning system and the other called the control system. The 
planning system is set up to outline the way we go; the control 
system is a vehicle to make sure we are going that way. What too 
often happens is that sumebedy will do the planning but nobody is 
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Figure 1. Schema for integrating system analysis and management systuns with project planning and control steps 
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and controlling ol projects 



really controlling anything, On the other hand, we have some project 
managers exercising control. They are worried about the budget, 
the schedule and so on but you look around for the plan and none 
exists. That kind of situation would be like just getting in 
your car and suddenly driving down the street not knowing where you 
are going. Both of these go together, If you plan something, you 
intend to control it. If you control something, then there should 
be a plan. 

What's involved in planning? Basically we identified five sub- 
systems. The first of these ve call project definition which is the 
outlining of the goal of the project, moving from large parts dovn 
to small parts. We use terms li-ke factoring, decomposition, analysis, 
disassembly. What you do is take a global task and try to break it 
out into its constituent parts. This is where your systems concepts 
become quite useful to you. 

Let's now take the technique of Program Evaluation and Review 
Technique in the right column of Figure 1. We can relate to PERT 
and show how it fulfills each one of the planning and project 
control ideas and also to systems analysis urd the vay we are using 
it here. In the PERT system, there is a major objective to be 
accomplished and you develop vhat we call a work breakdown structure. 
This is hierarchy factored just as it was over in the system column. 

Wc can eventually break out smaller and smaller tasks which we 
eventually call activities. We then take these activities and put 
them together in a flow graph which we now call a network. 




- 100 - 



10 5 



Where PERT differs from systems is that with PERT we now 



produce time estimates t * the job; we establish what we call 
slack conditions for management purposes. Then in the controlling 
function, you take care by noting the deviation that occurs between 
what should be happening as outlined by the plan and what is 
actually happening, taking those actions which are necessary to 
correct the thing and then recycling to get back by establishing 
a new set of ,r shoulds M or by revising the existing "shoulds". 

A Mod el of Pro j ect Management 

What I have done over the last several years is to study 
various project management systems and asked, "Why can r t a 
project director develop his own system?" If he understands what 
is going on, what the functions are, then he is not bound by one 
particular technique. One of the problems you have is that people 
can see the same global thing in different w/rys. What a staff has to 
b is agree on some kind of structure before they can move ahead. 

The main thing here is the outline of the task. ^'nce tasks are 
outlined we put them into a flow. Once we get that done ve go on 
to time estimation. Here is where you run into some interesting 
problems because, if it is a R & D situation, you are going to 
have to develop expertise with what we call probabilistic systems. 

You are going to have to recognize that unplanned elements can 
happen and there is a need and a way to protect ourselves on that. 
Scheduling is a matter of setting up the resources we need and w’hen 
we want them. This gets to be a real problem. 1 have seen a lot of 
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piojects run into trouble because the managers were thinking they 
could get a consultant at a certain point in time when they needed 
him but they had failed to check with the consultant. So they 
had to delay a ceitain vital part of the project until the consult- 
ant was available or else get a different consultant. In many 
cases a different consultant has not had che same quality and 
consequently the project suffered. So scheduling is a real prob- 
lem- Budget preparation comes last. It is done after the others 
are accomplished. Many times I have reviewed proposals in which I 
wonder if the seme people who prepared the budget prepared the 
rest of it. It is very hard to see the relationship between the 
work to be done, the objectives to be achieved, and the budget. 

Under control systems three points v T ere emphasized. The 
first point is that if you Lre going to control you do this 
throv'.gh a report system. There is an expertise just in this 
particular thing. I would say to you first of all reports have 
to be on time. It doesn r t do any good lor 3 man to know about 
problems too late ;o do anything about them. Second, they ha/e 
to be brief, and if at all possible they should be visual. They 
should be written up in graphic form so that the problem can be 
readily identified by the manager so he can see what lie has to 
deal with. Make it very simple, one or two pages. If you then 
want to supplement it with necessary appendix material, go ahead. 
Second, the report identifies the problems to which the management 
give his attention. Using problem analyses techniques, he works 
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out a solution to the deviation from plan. Thiid, having made his 
decision, the manager must majce sure it is implemented. 

Summary 

The general model presented covers the basic dimensions of 
project management in education. Proper understanding of the 
basic concepts and principles will give some assurance that you can 
be a successful project director or manager. 
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When I first joined the International Business Machines company 
I knew nothing about computers and couldn't understand what people 
were saying. Every second or third word sounded like M byte ri and 
"bit" and "index sequential storage". It was a completely different 
language that sounded like English but failed to make any sense. 

After I had learned a little about the business I recognized 
that the language was indeed English and made sense provided that 
1 s^ snt time studying it. 

The point ir the people who work around computers have a jargon 
all of their own, and unfortunately, this jargon is recursive; that 
is, if you don't know one term you are prevented from understanding 
by context other terms which follow. 

I would like to start today by skipping through a few definitions 
with you so that we may jointly have a common basis of unaers t anding . 

A glossary is by definition a list of terras in a special subject, 
or field, or area of usage with accompanying definitions. Gilbreth’s 
definition of technology is the most satisfying I have yet encountered 
The systematic application of scientific or other organized knowledge 
to practical tasks. We use a similar definition for education tech- 
nology is the application of technology to education, i.e. the 
application of an organized body of knowledge (a technology) to a 
practical task, in this case education. 
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Instructional technology is a subset of educational technology. 
Here we are talking about that part of education which is known as 
instruction. Instructional technology is the use of technology, 
the use of organized knowledge, and in this case the practical 
task is to improve the student’s interaction with his materials, 
the subject matter, and his teachers. 

Instruction is: interaction between a student or learner and 
(1) learning materials, (2) subject matter, the discipline itse'f, 
and (3) the instructor, the individual who is available to interpret 
and help him understand, his teacher. 

If I take that same definition of Instructional Technology but 
change a couple of words 1 have a useful def 1 tion of Computer 
Assisted Instruction. If we cross out tn words "instructional 
technology 1 ' and replace them with "CAI", instead of speaking about 
technology generally; which would include such things as audiotutors, 
video tape records, books, films, or tape recorders, these other 
tools; we talk about the use of a computer, then we have: CAI is the 
use of a computer to improve the student's interaction with his 
materials, subject matter and teachers. 

As we begin to look at computer assisted instruction specifically, 
and before we get to the question of how one goes about authoring 
material for computer assisted instruction, we need to look next at 
the varieties of computer assisted instruction I will present these 
in a few groups and try to make some differentiations between these 
uses of computers in instruction. 
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DRILL 



PRACTICE 

PROBLEM REVIEW 

DIAGNOSIS AND PRESCRIPTION 

TUTORIAL 

GAMING 

SIMULATION 

FACT FINDING 

COMPUTATION 

LOGICAL PROBLEM SOLVING 
EXPLORATION 

Figure 1. List of different ways computers may be used to improve 
instruction 



There is no real hierarch in the list; that is, although the 
first item is called Drill and the last item is Exploration, this 
does not necessarily imply that the one application is any more 
useful, any more powerful, than the other. 

A computer may be programmed to drill a person on requisite 
knowledge* It may be programmed to permit a person to practice the 
use of skills or techniques or it may be programmed to provide 
the student with assistance in problem review. These are different 
applications of the computer. 

A number of fairly large projects with a lot of money involved 
have been less than successful because of the failure to understand 
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the difference between Drill and Practice. And yet, when you sit 
down and think about it, most any classroom teacher can specify this 
difference to you. As an example of what I mean by Drill and what I 
mean by Practice, consider some skill involved in basic arithmetic. 

If I ask you "What is 9 times 9? M I expect a responds of M 81"r I 
don’t expect a bunch of manipulations with numerals which lead to 
the answer 81. And my criterion performance of good behavior would 
be how quickly you emitted the correct answer and in terms of overall 
performance how frequently you emitted the correct answer- 

Now in programming drill fur a computer you write a program 
which will (1; include randomly all possible combinations of numbers 
for which the student is expected simply to know the answer. And 
in presenting such a program to the student, you are not much 
interested in telling why 9 times 9 is 81. The concerns are (1) that 
the student give the correct response an 1 (2) that he give it quickly 
and consistently. 

Practice < n the other hand can be illustrated by the Problem 
99 times 9° In this case the student is expected to (1) know 1 the 
numt r fact 9 times 9 and (2) know an algorithm or procedure for 
multiplying two digit numbers. The exact response and the speed 
with which it is made are relatively unimportant. 

Drill and Practice are quite different, and you don't program 
remediation for them the same way. If an error is made in a Drill 
the correct remediation is to represent the problem with the correct 
answer supplied. If an error it made in a Practice, the correct 
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remediation may be either to teach missing number facts or to teach 



the proper procedure. Additional diagnosis must be done to determine 
which . 

Problem Review is another* rather different yet associated, use 
of computers in instruction. This is a situation where you may have a 
student, say you are teaching him basic science, and you ask him to 
calculate how quickly a body falls from the Leaning Tower of Pisa to 
the ground. The student is expected to have sat down and independ- 
ently calculated the answer and figured out how to solve the problem. 
Then when he sits down at the computer terminal, you ask him a 
question such as: What answer did you get? If he got the correct 

answer you might tell him this is right. In case of error you might 
ask him to type the formula he used in getting this answer. 

In other words you would do the same kind of interacting between 
the student and the computer, as you would in a classroom if you 
were sitting down working with one student rather than a whole 
group of students. Going over a student's individual problems at 
a terminal is an effective use of the computer as instructor. 

The concept of using a computer to find out what a person knows 
and what lie needs to know is very attractive. The idea is that you 
place a learner at the terminal, give him a task to do, look at the 
answer, watch the way he performs the task in terms of building 
sentences or doing problems or whatever, and then tell him on the 
basis of what you find out that his response is correct or s^nd him 
to "other" instructional tools in order to practice the needed skill. 
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instruction, CAI as veil as video tapes or films. 

Tutorial is the most common concept, the most commonly understood 
application to student instruction. This is the kind of situation 
where in essence you have put a programmed book on a computer which 
has the ability to understand natural answers. There you are 
engaged in a dialog with a student except that you are not really 
there. You are there by being the author of a program. The student 
interacts with a program rather than with you directly. 

The tutorial, concept of CAI is the oldest of all the CAI 
concepts. When Ralph Grubb and others first began to work with 
computers and instruction in 1958 or 1959, they wrote tutorial 
interaction so that a typewriter would ask a q"e^ti.on of a student 
who would type back an answer, and the typewriter would respond on 
the basis of the answer provided. It is quite difficult to write 
effective free language CAI instruction. 

It is possible, of course, simply to put 1 - Skinner- type 
programmed instruction programs and let the rat nine keep track of 
the student's answers, but I do not believe there are benefits in 
doing so. Why use an expensive machine as a page turner? 

One of the real problems of CAI, is that it is so exciting 
when you can program a computer so It sits up aid whistles, as it 
were, that people tend to write programs simply because they are 
fun to do, not becuc.se they fill a need, Co* .p assisted instruc- 

tion is as old as PI; yet ev^n at this stage, 10 or 11 years later, 




we are still in the process of trying to understand what there is 
about the computer in instruction that adds something that other 
techniques or tools do not. It simply does not make sense to write 
a CAI program if you can do the job just as well with a motion 
picture or printed page. 

Gaining and simulation might be regarded as being at the opposite 
end of a continuum; in other words, there is a fuzzy dividing line 
between what is gaming and what is simulation. I think the most 
useful distinction between tl em is that a simulation is an imita- 
tion of reality whereas gaming is something one does just for the 
fun of it. Yet, even gaming can be used as : rather powerful instruc- 
tional tool. There are a number of computer games for which one must 
have command of certain basic skills to be successful, such as spelling 
in some word games. There are games you can do which are basically 
fun but which require you to develop speed; in this casa, it would 
be similar to Drill to develop quickness or the ability to apply 
skills in which they become similar to Practices, 

Simulations, on the other baud, are the bringing of reality 
into the classroom or onto the terminal so the person can compress 
things in terms of time. For instance, one of the oldest computer 
games is the Sumerian game in which the student fulfills the role 
of the warrior priest king of ancient Summer. He has to make decisions, 
such as how many people shall plant Lhe grain needed for food, how 
many people will become soldiers tc guard the frontiers, how much of 
the total national resource should be put into killing rats so they 
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don't eat so much of the grain, and should they build rat-proof 
warehouses which will cost $10,000 each or wooden warehouses which 
cost $5,000, or do they simply sustain a certain percentage loss 
to rats, in sun, all kinds of decisions which have to be made in 
order to live in a community. 

There is no possible way of providing such experiences first- 
hand for a learner, yet on a computer we can compress the time scale 
and the learner can practice making decisions. A medical srudent 
can practice diagnosing and prescribing for an ill patient. If 
he makes a mistake the patient dies. You can't let a student 
do that in reality; yet, with a computer, he can readily gain skills 
in these directions. 

Somewhere between doing things for the fun of it and in imita- 
tion of reality are items w T hich can be found along the continuum of 
games and simulation. 

Another simulation is practice for job interviews. The computer 
asks a question; the student gives 3n answer; the computer asks 
another question; this situation is very similar in nature to the 
diagnostic materials which are used by the medical people. Yet, I 
think this would be very useful in the work with the under-educated 
people yu have here at the Center. 

Oddly enough, one of the powerful uses of the computer is simply 
to find facts. Library skills are important but once you have 
developed library skills you still have to go to the library and 
spend much time to find Information. Perhaps the computer can be used 
to help you find information quickl> and easily. 
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Interestingly enough, Computati on is an instructional use of 



computers. Indeed, in the entire field of computers in instruction, 
there are probably more terminals being used lust for computation 
than fcr all the other uses combined. A number of vendors offer a 
service called Basic, for which you put a terminal in the classroom 
where the student simply sits down and computes on it. 

Logical Problem Solving, as opposed to Problem Review, is this 
idea of critical thinking, the process of learning how to take 
advantage of facts and ideas in order to arrive at solutions for 
real tasks. People who are concerned with critical thinking and 
problem solving talk about hypotheses and testing hypotheses. In 
ordinary English this comes out like this, "Well, let's see, that's 
the problem; v^hat are the possibilities?" And so you think about 
the different ways that it might be explained. U I have so many 
dollars; I have to pay so much for rent, so much for insurance, so 
much for food; what is the optimum solution in terms of relative 
applications of resources?" You can try out solutions and get 
answers, then change variables and get different ansvtrs and so a 
handle on what would happen -- until you arive at the kind of solu- 
tion that would be applicable and make sense to you. 

The last concept I would like to introduce is the idea of 
exploration. This probably has more future and more promise in 
terms of the use of computers in instruction than any of the others. 
This is the idea of making available to the learner, in essence, all 
the accumulative knowledge of mankind, in a form that he can go and 
get as he needs it. 
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As I think you can see, from the list of items we have gone 
through, there are a lot of different ways of writing CAI courses. 

Some of them, such as programmed instruction on a machine, consist 
of no more than just putting in a series of sequential frames for 
the students to see, with the following frame basically determined 
by his rnswer to the previous question. Others, such as Exploration, 
involve somewhat different organizations of knowledge. Let's look 
a little further at Exploration. Figure 2 (next page) might be a 
model of the knowledge involved in some subject matter. The illustra- 
tion is diagrammetic and has no reference to reality 

As you look at any subject matter which you want to teach people, 
you can define pieces of knowledge, Some of the pieces are relatively 
small, and some are relatively large in terms of what you have to 
know, understand, and relate in order to get an idea down firmly. 

I am sure that of the learners that come to your classes, each one 
brings to any experience, be it learning or any other, a widely differ- 
ent 1 ackground, and what they know is also different. It doesn't 
make sense to me, then, that everybody that comes to a classroom 
should be taught exactly the same thing A person coning to a class- 
room should learn just those things that be needs to know that he 
doesn't already know T . When you set up your criteria, your objectives, 
and specify critical behaviors for a body of knowledge, what you are 
really saying is when a person finishes studying this material this 
is what he will know or this is what he will be able to do, I don't 
think it matters at all whether every individual who comes to study a 
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Figure 2, Or gantza tiot. of the knowledge available within a subject area 
dotted lines show paths followed by two different students 
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body of knowledge reads every page of the book, or listens to every 
void of the lecture, or covers all rhe material in exactly the same 
way. What is Important is that the person, when he finishes, is 
able to meet the course objectives. 

One of the real promises of CAI is the ability to permit a 
person to study those things he needs to know. Each student should 
be able to go at a rate that is comfortable and meaningful to him 
as he progresses. In a given body of knowledge, a student may 
enter the course, study this material, then go study that material, 
be satisfied that he knows what is involved; and go take the exam- 
ination or otherwise attempt to meet the criteria. Some people are 
very organized and w T ill take first this and then that, other people 
may explore widely. Yet each one is equally happy, and gets equal 
effectiveness from the material , I maintain that this is the proper 
way of organizing material for instruction. 

Let’s look a little bit at the problem of writing and authoring 
materials for CAI. Figure 3 (next page) compares the writing of 
materials for CAI with the w’riting of any other sort of instructional 
materials. In either event an author is involved; he simply writes a 
manuscript, and when he finishes a chapter, page or unit, he then 
submits this material to an editor for review, revision, suggestion, 
and correction. This is a regular procedure, for there is no one of 
us whose written output cannot be improved by the assistance of 
some other person looking over our shoulder and pointing out to us 
where we made assumptions that were not valid, and the like. 
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GALLEY PROOFS 



\ 




J L 



PRINT 



PAPER MANUSCRIPT 



1 




ON LINE DEBUGGING 




DISK FILE PAPER MANUSCRIPT 




Figure 3. Comparison of conventional and CAI authoring procedures 
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In the usual publishing process, once the editor and the author 
have finally agreed concerning their material, the editor then sends 
the manuscript to the typesetter who is a specialist. The author, if 
he wants illustrations, draws stick figures or writes a description 
of what he wants illustrations to be, and the editor sends the manu- 
script off to an artist to have them done properly- After the type 
is set and the plates are made, the whole thing goes to the printer. 

Up to this point, a paper manuscript exists; from here on, a typescript 
will exist. 

The output from the printer goes first to a proofreader, a 
person who checks to see that spelling and punctuation are correct; 
then the galley proofs go back to the author and editor for review 
and any necessary changes. Once the text is in print it looks 
different. Eventually, after everyone has approved the sample 
pages, a production run is m~de and the book exists, or the set 
of slides exists. 

Now let's look at the CAI process- When textual material is 
to be used on a computer, the autnor's task is exactly the same 
as when he is writing for any other media. Just as an editor in 
the publishing business must know about typesetting, even though 
he himself may not be a typesetter, and must know about what is 
possible in the printing process, although he himself may not be 
a printer, so an editor concerned with CAI has to understand 
what is possible within the particular computer system that is 
being used. 
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Eventually a manuscript, a joint product of an author and an 
editor, is sent to a keypunch operator, whose function is to punch 
the data on cards so that they can be printed and entered into the 
machine. From the keypunch operator the finished cards go to a 
coding editor, a person who understands all the vocabulary used in 
computer coding. When the coding editor finishes, he sends the 
completed deck to a machine operator who loads this program to 
the machine. The debugger means exactly the same thing as proof- 
reader. Only after all these people have done their tasks does 
the program in the form of an on-line program go back for the 
author to take his own course to see how it looks. The author 
does not need to know the machine language. Finally, when all is 
done, a new course is available. 

When you are writing for a printed page, a long paragraph can 
be continued on the next page. When you are writing for presenta- 
tion by computer, if you are using television-like screens to 
presen'' the material, only so much material can be put on one 
screen. If you are using a Teletype, or if you are using a type- 
writer, the machine prints so slowly your concern is for the 
economy of w’ords. In either event, an author writing for CAI 
must be confined. We have found this form to be very useful 
(Figure 4 - next page). The author and his editor are concerned 
with the instructional integrity cf the single frame presented 
to the student at one time. They try to get one idea on one 
frame. Convention specifies that every frame must have a name, so 
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when you write for CAI the concept of naming things is important. 
You name every frame that you present. One way of naming things is 
simply to number them. 



When a frame shows up on the CAI system, the page number will 
be called a label and will be the identification needed by the 
machine to enable it to present the frame to you when you ask for 
it. 

After the student has seen a given frame, there are several 
things that he might do; for example, he might go on to the next 
page. In this case, all the author needs to do is specify by a 
simple check that he wants the student to see the next page after 
he has finished with this page. If the student is to go to a 

different page, all the author does is indicate the name of that 

page in the space provided. But if the student is to go to a frame 

that he is to select, then the author indicates that the student has 

freedom to make his own selection. 

On the other hand, it may be that the next page the student 
sees will be determined by the answer ht gives to a question on 
the current page. If that is the case, the author provides at the 
bottom of the form a decision table. This is basically a little 
chart that tells the machine to send the student to page ,r X n if 
his answer was this, or page T 'Y 1f if his answer was that, and so on. 

This is a general authoring system for any CAI system. Notice 
that, in order to use this system, the author does not have to know 
a computing language. All the author has to do is understand how to 
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present ideas to learners of the age, ability, and skills that will be 
using the material and how to think logically, It is the concern of 
the coding editor to get this material into a form acceptable to 
the machine being used. 

In addition to the written presentation on the screen, and the 
decisions concerning the answer, there may be ui'sociated with each 
frame audio messages to be played, and if it is audio, you specify 
the name of the message. He might also want to see given frames on 
a projector, images which are prepared for presentation, color photo 
material which is best done on a projector as opposed to a CRT type 
screen. Notice that the decision table permits the learner to go 
to this frame if his answer was this the first time he gave it, but 
go to that frame if his answer was this the second time he was given 
permission to answer. 

The output from one of these frames is a manuscript consisting 
of many pages. An advantage of a paper manuscript is that it 
defines what is going to be seen on the screen; it shows what will 
happen on the bauic of any answer a student gives. 
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MU LT I -MED I A P ROG RAMMING 



William A. Deter line 
General Programmed Teaching 
Deterline Associates 
Los Altos, California 

The technology of instructional programming grew up tied to 
the printed word, in the early teaching machine presentation 
devices, and in programmed textbooks. Well over 95 percent of 
all programmed materials presently available still take the form 
of programmed printed materials. There is an obvious conclusion 
to be drawn from this: programmed instruction is a "book-bound" 

method of instruction. But that conclusion is not correct; instruc- 
tion-including programmed ins true tion--can take many forms, and 
c n utilize a variety of media. 

Many articles have described programming as a "process". 

Actually we should make a distinction between two processes: the 
instruction itself is a process, interactive in form, requiring 
active responding by the students, and providing confirming or 
correcting feedback or evaluation; and the design and development 
of the program is also a process, a process that requires objectives - 
the operational definitions or specifications of the desired learning 
outcomes - and an empirical developmental process of testing and 
revising the instruction "o make it achieve the objectives. These 
two processes are as applicable to any other medium as they are to 
the printed word. 
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When we look at media from the instructional design point of view, 



we should do so with objectives in mind, and ask ourselves what kinds 
of stimulus inputs can be used to facilitate, clarify, and make more 
comprehensible either the responses themselves, t c the stimuli to 
which the student is to learn to respond. Although we cannot go 
into the complexities of this approach, it should be clear that this 
is quite different from the more typical and more casual approach 
of selecting a variety of media on the basis of relevant content 
and availability! Typically the rationale for construction of 
"multi-nedia courses 11 is the principle of redund ancy . The teacher 
tells and shows, and assigns text and reference coverage of the 
same material, and then brings in a variety of audiovisual presenta- 
tions to present the same information all over again in a different 
form. This is generally a "shotgun" approach that has very little 
systematic or professionaJ expertise applied to it. The approach 
says, in effect, "Since no one approach seems to do the job for all 
students, we will subject all students to all repetitious presenta- 
tions, hoping that as a result, they will ail learn." Tins is a 
part of the basic instructional approach that lias always dominated 
public education, an approach that concentrates on content and infor- 
mation presentations as the design emphasis, rather than on an 
approach that concentrates on changing b_ohavio£ by specifying that 
behavior and then building whatever content and instructional inter- 
action that an empirical approach finds to be necessary. 
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It is possible to program any ins true t ion, whether that instruc 
tion is presented by a teacher, an audio tape or video tape, a film, 
or a textbook or workbook. But the programing process - at least in 
its initial stages - is the same, independent of the media that might 
eventually be selected. The objectives must be identified, ar.d so 
must the entry level competencies of the students, in relation to the 
objectives, and in relation to the media selection possibilities. 

We need to ask what stimulus inputs are relevant to the objectives, 
either as stimuli to which the student must learn to respond., or to 
teach the student about the stimuli and related responses. And we 
have to consider the capability of the student, and whether a given 
medium will facilitate or hinder. Practical considerations are also 
important, those considerations of convenience for studer . and for 
the teacher, as v:ell as cost, and equipment availability and operation. 

The "language lab" is now a fairly conimon application of audio 
tape for instructional purposes. I spent a year working on a research 
project at the Defense Language Institute's Language School at Lackland 
Air Force Base in Texas. One of the very first language laboratories 
was developed there a number of years ago, and the decision to employ 
audio tape for language instruction was a perfectly sound one. At 
the Language School, English is taught to foreign stude: ts - English 
as a foreign language. The School does not have instructors who can 
teach foreign students in thoir own language. All teaching, explain- 
ing, and instructing is done in English, building on the very slight 
proficiency that the students bring with them. It is a fascinating 
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structure of instructional design problems 



Audio tapes provide 
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drill and practice, and, in some oi the newer materials, language 
development itself. Audio tapes - particularly m cassette form - 
provide a potentially powerful instructional looI, not only for 
language instruction, but tor instruction m many kinds of other 
subject matter areas. Hut most language laboratories are only 
partially effective, and very inefficient, and some of the 
materials used are almost unbel iovao ! y bad! The medium is not 
so powerful as to be able to make a poorJy -'esi^ned or irrelevant 
"instructional exercise" into a highly effective and efficient 
instructional tool. It should be obvious that the same is true 
of films, television, books -and lectures. 

Research and development and empirical coma 1 development 
projects have indicated clearly that intuitive, logical, a priori 
media selection and design are simply not adequate- ’l he* inade- 
quacies are not as readily identified in the more conventional courses, 
since it is a ssumed that the course structure is adequate, and that 
all inadequacies identified by course examinations ate in some way 
the fault of the students - and the students pay for it But tie 
empirical approach directs evaluation attention to the components 
of instruction as well as to the students, and revisions of mate- 
rials can eliminate the bulk of the inadequacies - oi which there 
are many - previously blamed or the student - 

Multi-media courses, and mult 1 -mod in programs for segments of 
courses should make use of the empirical development modi' 1 -eel 
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specifications of objectives. This does not mean that when that 
approach cannot be fully implemented that the old ''redundancy 
principle" and the "interest value and change of pace" principles 
are inappropriate. Sometimes we have no choice. Sometimes 
redundancy is the only available expedient, and multi-media variation 
can make c < rses more interesting. But since we can be more precise 
and more effective, and more relevant in our use of instructional 
media, those shopworn "principles for media selection 11 should be 
second choices, and more accountable methods should be used whenever 
possible . 

Individuals involved in the empirical development of instruction, 
whether for "live" instruction, 'packaged" instruction, ir education, 
or in training, have utilized flexible programmed lectures and pro- 
grammed demonstrations, programmed laboratory work, and programmed 
independent study. It is unfortunate that the computer and its 
terminology has led to an unfortunate interpretation of the word, 
"programmed 11 . Many people respond emotionally to the word, ' hink- 
ing of it as referring to something inhuman and inhumane, mechanical, 
stifling, and contrary to the values of creativity, intuition, and 
insight. In the instructional sense, programmed instruc Lonal mate- 
riajs and present at iens are those that have been designed to provide 
interactive instruction, to achieve specific objectives, and do so 
reliably and predictably. The do much more, and well-developed pro- 
grams can stimulate interest, curiosity, and creativity, and develop 
competence, and self-c* nf idence . Other kinds of instruction can also 
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do those things, but neither conventional nor programmed instruction 
will always do it unless creatively designed and implemented. And, 

I believe, this requires accountability through all phases of design, 
development, and implementation. 

Implementation, surprisingly enough, is not our strong suit; 
that is, the plea for more precise specifications in the form of 
objectives does not in any way imply, "We know how to teach; what 
we need are better specifications." The bulk of activities called 
teaching are little more than pres entation methods , except in the 
lower grades, where interactive instruction is a necessity, or 
nothing much happens (actually, it is a necessity at all levels, or 
very little happens, and accountability won't work in an environment 
in which teaching is no more than bombarding the students with 
information). Elaborate or simple presentations and performances 
based on only vauge notions of the kinds of competences to be 
developed in the students, confounded by a failure to direct and guide 
student interaction, appear to be a rather unprofessional and casual 
way to go about something as important as instruction! We do very 
little to improve our chances - and our students* - from that point 
on. Frequent progress evaluation and remediation might help save 
it all, but, again, there is very little of either except at the 
lower elementary level. We don't seem to care about finding out 
what is happening to our students on a day-to-day or even a week- 
to-week basis. Tests that might help us diagnose student difficul- 
ties often are so widely spaced as to leave the students In a 
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cumulative failure position for weeks or months before the diag- 
nostic data are collected. But then the diagnostic data are not 
used for diagnostic purposes - not in any systematic manner - 
nor is much, done that could be called remedial instruction. 

Described in those unflattering terms, our instructional imple- 
mentation procedures are not very impressive. 

The term, "multi-media programming is of temporary value 
only, to refer to the programming process applied to the develop- 
ment of courses of instruction involving a variety of media ana 
methods. But it is the programming process — not the multi-media 
aspect -- that is the more important: Objectives, interactive instruc- 

tion, and validation (testing and revising until the course achieves 
its objectives) can be applied to any medium, any method, and to 
any combination of media and methods. )he implementation methods 
required to make each component of instruction do its specified 
task might be the single most important development in education 
during this period of innovation and change. We have learned how 
to do it in small, special projects; now all we have to do is 
learn how to do it on a broad front and on a large scale. 
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A MODEL FOR PRESENTATION DESIGN 
AND MEDIA SELECTION 



Donald T. Tosti 1 
Vice President 
Individual Learning Systems 

In the early days of programmed instruction we naively believed 
that the produce we were developing was so innovative and so powerful 
a learning device that as soon as it got into the hands of the stu- 
dents they would cry for more. Any means of reaching the student, 
even by door to door selling would lead to an overnight revolution 
in education. So we thought. We were not alone in this supposition, 
for by 1963, something like 200 teaching machine - programed learning 
companies had been formed. One year later, most were bankrupt. All 
of the bright, eager advocates had neglected one item - we never 
really questioned how PI material would be used in the classroom. 
Failure was rationalized. It was fashionable to blame the classroom 
teacher for being resistant to change and fearing replacement. But, 
the negative reaction of teachers stemmed not so much from a fear 
of being replaced, but rather from a reluctance to implement some- 
thing which did not really fit in the present educational system. 

The resulting "bust'’ in the PI business produced many casualties 
However, a few professionals still clung to the belief that behavior 
psychology could contribute to education. Some continued to write PI 
others wrote on how to write PI. And some of us began to see if we 

^Formerly Division Manager, Behavior Systems Division, Westing- 
house Learning Corporation. 
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could design methods of incorporating our materials into the exist- 
ing or modified educational system. For this latter activity some 
useful information had been collected resulting from the testing 
of our previous programs. Pi fortunately allows one to observe 
each step of the learning process. So, we began by taking these 
observations and generalized them beyond the tight stimulus control 
of the frame to broader concepts of an educational system. Individ- 
ually prescribe instruction, contingency management, and educational 
accountability are examples of these extensions. Most recently this 
research has led to the development of the media or presentation 
design model which i will now discuss. 

We are at the present time testing this model at the U. S. 

Naval Academy under an Officer Education Grant. It is a significant 
research project which requires well over a million dollars in 
its implementation. Its objective is to determine exactly what 
effects various educational media produce. An important collary 
to this question concerns the cost-effective considerations of 
methods and media. 

We first had to come to grips with the problem of defining 
what educational media are. An examination ot the research on 
media is very disappointing. Attempts to classify lactors that 
could be tied to media was almost hopeless. It appears that those 
educators who favor certain modi'’ do so more for emotional, reasons 
than because of any research results. In tact, the research on 
media is so disorganized tl «v anyone who is trying to put together 
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a multi-media package is almost forced to rely on personal preference. 
We all talk about the grand advancement of multi-media and mixed 
media educational systems, but there are no rules for mixing media 
or for the design of multi-media courses. This state of affairs 
led us to attempt a different approach. 

For this, my engineering training came in handy. A medium in 
an engineering sense is thaL which transmits something. Every 
medium has limitations that modify your delivery some ways. For 
example, the diameter of a water pips acts to limit the flow at 
a given pressure. In engineering, we examine the limitations - 
not the advantages of a medium. Might we not examine educational 
media in a similar fashion? We may assume that any educational 
medium limits us in some way. It ,r cuts off" in certain predictable 
ways some aspects of the message. The "message" in education is 
not just the content but includes the form of the "package" as well. 

Let me Illustrate the point. Let’s consider this presentation 
form: (1) Information is to be presented in a verbal or illustrated 

display, (2) a demand of a written or selected response is to be 
made, (3) the presentation will last as long as the student desires 
it, and (A) a student must respond before going on to the next 
item. It is important to note that I did not specify any educational 
medium. Although certainly the choic2 of media we may use is 
effected by the presentation form. As you may recognize, this presen- 
tation form is a standard PI form. It usually is implemented via a 
work book, but it doesn’t have to be. The PI p» 1 ■» h ive become very 
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inventive and have used other media sue' as slides and tapes to i- e 
ment essentially the same presentation 

Let's look at that presentation when \ce try to put it through 
the single medium of motion pictures, We find out that condition 
(1) can be met, but condition (2) is not since there is only a 
covert observing response demanded in response to this medium. 
Condition (3) is violated - the dur. 4 ion of the presentation is pm 
set, is not available as long as tl student desires it, Since tl.-. 
next presentation occurs no matter what the student doe:-?, condition 
(4) is not met. This example clearly indicates what we mean by 
limitations inherent in the medium. A movie cannot convey this 
presentational form intact. 

Let's take a more sophisticated device - the human tutor. 

A human tutor alone cannot present stimuli in anything but a 
verbal form. He can demand a vocal response, because he can 
haar and the student can speak. The presentation lasts only as 
long as it takes the tutor to speak the words so there is no way 
the student can linger over these stimuli. Condition (4) can be 
met by the tutor demanding that the student must make some 
response before proceeding. Clearly, the tutor is also limited 
in presenting the PI presentation form. If, however, instead of 
the tutor alone we use mixed media presentation which involves 
the tutor plus the blackboard and a writing tablet, we can 
convey our original presentation form intact. 
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Now what are the Important charactet is tics of the important 
presentational forms? As I said, we first attacked this problem 
by listing every factor v;e could think of. This included attitude, 
atmospnere 5 color of the walls and more than 200 other factors. 

We soon decided we had enough so we began to analyze and condense 
them. We finally narrowed the factors down to six which we felt 
directly influenced learning. We found these six dimensions formed 
three classes: (1) stimulus class, (2) response class, and (3) what 
we call an instructional management class. These classes vary in 
two very important ways; in form and in frequency or rate, The 
dimensions thus appear as follows: 



The Dimensions of Presentation 



Stimulus 



< 



Display F or m 
Environmental structure 
Fictor ial 
Symbol ic 
Verbal 



Duration 

Transient 

$ 

Length of tine the 
./presentation >*emai 
] in tac t 

Persistent 



Resp onse 
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Response Demar. d 
Covert 
Selective 
Constructed 
Vocal 
Jotor 

Affective 
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(Frequent 
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Sparse 
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i ' 1 1 e_ in o ns i o ns o f Prese nta^ion ( c on t inued) 

M- i ia g e r Purp ose 
Need Management 
Achievement Management 
Prescriptive Management 

M n na gement / 

] Enrichment Management 

Motivation Management 

Systems Support 

Although this structure may appear logical and simple, it took almost 
a year and a half before we saw these relationships coming out of 
the mass of variables we began with, (To s t i and hall, 1969). 

T he fi rst d imension , the display form or stimulus, can be 
represented in a variety of ways, but four forms are most common in 
education: (1) environmental structure; (2) pictorial - any illus- 

trated information at all; (3) symbolic - as in matheriat i cs , and 
(4) verbal, both written and oral. Now there are obviously other 
ways in which information can be displayed, (hie could use gestures, 
odors, feel, etc., but usually these forms are unin poi tan c in 
education . 

Suppose we decide a picloral display is the most appropriate 
presentation form. We have not specified what medium we right 
use to convey that picture. Obviously, a book could handle a 
picture, so can a film or a blackboard - which one do we want to 
Perhaps, if we consider the ether 5 dimensions tht choice 
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Inst i uctional Management 
Fr eq ue ncy 






F requeue 
$ 

Frequency of decision to 
change presentation 

A 

Spar so 



use ? 



will br ' jrrowed. If our educational system indicates an ideal 
condition we would like to have for each of these six, not fur just 
the display, v r e .an begin to make sorre rational decisions on what 
media-mix we might use. Possibly we might be forced into tvade-offs. 
For example, a film may be too expensive, a book too dull, and a 
blackboard too inconvenient. So, it might be best if we could 
eliminate the pictoral display. If we break the verbal informa- 
tion down ' ore and thus increase the response demand frequency, we. 
might do just as well. 

Let ns now examine the se cond d i mension . Displays vary in 
their duration; that is, how long they will remain intact. The 
impact on learning of duration can be very important. Display 
varies from very cransient to very persistent. Various media have 
different capacity in i.andl ing the duration aspects of the display. 

For example, a book cannot handle a highly transient display. While 
a movie normally handles a very transient display with ease. Movies, 
however, can also handle persistent displays. For example, there 
are underground films that show the Empire State Building for 18 hours. 

Duration factors are frequently found, hut not identified as 
such in Educational studies. Researchers have often compared the 
highly transient display of movies with nixed transient- persistent 
displays of conventional lectures nr they have compared conventional 
lectures with the still more persistent FI. Inherent in all ttr»se 
experiments has been this confounding factor of delation. It is 
difficult to compare PI with lectures warn the student, at his 
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a leisure, can observe the PI display ar long as he likes. Such research 
is not analytical when as potentially powerful a factor as duration 
is not being controlled. 

Too much research is of this form. One of the major considera- 
tions of the AnnapoJ is project is to eliminate the confounding of 
m^dia with presentational considerations. There are mar,.* important 
questions raised. Is duration a factor in learning complex informa- 
tion? Khat are the interactions between presentation dimensions and 
what we are teaching? Or the kind of students that are learning, 

he t us nov consider the two response dimensions. There are a 
class of media that we usually neglect in our research which W3 
might ■ ’ll the response accep tance media. These are media which 
can accept student outputs and possibly store or transmit these 
responses. Such media may be just as important as display media. 

The note pad you are writing or» is a response - acceptance medium. 

My ear is a response - acceptance medium. Now* one way in which I 
differ from a video tape is that as a device I have potential of 
accepting oral responses, while a video tape has ro such potential, 
hven though our display charatt eri s t ics may not differ nuch, our 
response acceptance characteristics do. Ihis nay be an important 
difference between the lecturer and the IV, but only if the capacity 
is used. To a lecturer with SOO students in his class, his response 
accepted capacity is of little use. I'nfor Lunate! v , in education it 
is as common for us to pay for unusable media advantages. 
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It could be, and we are investigating this now, that the most 
significant single dimension of presentation is the response demand 



frequency. There is some evidence that the success of PI and CAI 
does not result from theii elaborate feedback mechanisms, or their 
precise attempts at establishing stimulus control but, just from 
the high frequency with which they demand response, which causes the 
student to interact with the material. Our preliminary results indi- 
cate that if you can guarantee tnat the stulent is interacting with 
the material, the frequency with which he is required to respond 
may be the best predictor of success. It may also be the. factor 
which is most sensitive to individual dif Eerence.-: in students' 
abilities. We suspect, and there is tentative data on this, that 
some students require a higher response - demand frequency than do 
other students. Most of us in this room have bean able to learn 
successfully from looks, which have a relatively low response - 
demand frequency; however, we know of others who require much higher 
response - demand frequency. 

We have in out own work developed instructional material which 
we call multi-level materials, which art really presentation variations 
in response-demand frequencies. The top level is usually a textual 
or motion picture p risen tat ion with a very low response-demand. The 
student may read three or tuur pages of material and then take a short 
examination on this information. Then, if he Is successful, he 
goes on to the next topic. If, however, he is not successful, he 
branches down to a greater level of response - demand frequency, 
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with information presented this time in a paragraph form with 
questions after each paragraph. If he is unsuccessful at this level, 
he is branched down to a still lower level with the high response - 
demand frequency in the form of PI frames. 

I must point out that response demand frequency is independent 
of any feedback consideration. Though feedback is often incorporated 
in such procedures, feedback falls in the 3rd category of instruc- 
tional manngemen c . 

There is a third class of media which we call instru ctional 
management dev ices. There are devices in any learning situation, 
which asses information end TiK.ke decisions to change the presenta- 
tion for a student. In conventional education, die teacher is an 
instructional management device. She listens to questions and 
changes her delivery as a function is assessing th<' student's needs. 
Instructional management can be very important in an educational 
system. We define it as the decision to change the presentation 
as a function of some assessment of the student's needs aspirations, 
or achievements. 

As a lecturer, I have a preset display plan. If I were stopped 
by a question from the clajs I might change my presentation as ,? 
function of that question, If I had previously recorded my planneo 
display on a video recorder, the system could not accoiimodale such 
instructional management. The video tape is limited in its ability 
to act as an instructional management medium. 
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Instructional management requires three procedures, (1) appraisal 
of the student, (?) selection of activity, and (3) initial . on of that 
action. These activities take place in every good instructional 
situation. We have gone into conventional classrooms and evaluated 
ongoing teaching in terms of what student data the teacher is accepting, 
what kinds of decisions she is making, and what kinds of actions she 
initiates. We find, for example, that in highly engineered class- 
rooms, there is ^ significant correlation between Lhe kinds of 
instructional management action that the teacher takes and resulting 
student performance. It appears that the more the teacher overrides 
the system, by deciding tilings for the student which he can decide 
for himself, the poorer student performance is. 

Although all forms of instructional management requires the 
same activity of appraisal - selection - initiation, what data to 
use and what activities to initiate depend on the purpose of m ana ge- 
ment , We have tentatively indicated five purposes three of which I 
would like to discuss now. 

The first purpose is called need m anag emen t. Tills form is 
particularly important in retried ia programs. It is necessary to 
be able to decide what curriculum is required to meet a particular 
student's needs. In the Job Corps system, students are given an 
achievement test battery to place them in the appropriate curriculum. 
Instructional management thaL allows the student to skip the next 13 
ftan-es if h> is familiar with the topic is an example of need manage- 
ment . 
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Purpose two, achievement ma nag ement , is the one which is most 
commonly employed. More effort goes toward achievement management 
than any other form. The purpose of achievement mar. age me nt is to 
insure that the student masters the desired objectives. There are a 
variety of ways of doing this. You may be familiar with individualized 
programs in which short tests are given after each presentation. If 
the student passes these tests he may proceed, but if he fails he 
is assigned different material covering the same topics. 

Another form of achievement management in a computer managed 
system evaluates the student ever/ two weeks. If he rails the student 
is given remedial instruction. In our Annapolis project, students 
are evaluated on a daily basis by means of a post-test. Each wrong 
answer requires a different form of remediation prescriptions. 

The confirmation technique commonly employed in PI is a form 
of achievement management. The confirmation of an incorrect response 
indicates to the student that he hud failed to acquire that informa- 
tion within the frame. It signals the student to engage in self 
management. He uses the data of hir "error" to re-examine the 
presentation and thus remediates himself. The author has found 
that in one situation more than &5/S of ail students errors could he 
remediated by this procedure. Thus eliminating the need for costly 
bi anches . 

Achievement management was not the original rationale given for 
using connrmation . Originally, we thought the confirmation acted 
as a reinforcer. However, we very soon found out that the confirmation 
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would not act as a reinforcor in the same sense that food could act as 



a reinforcer for an animal. It did not maintain behavior or material. 
With no other "reinf orcer" than confirmation, the student often gave 
up and threw the material away. 

The third purpose, motiv atio n ma nagement, is often confused with 
achievement management. We are not defining motivation management 
in terms of interest. Many studies have indicated that interest is 
irrelevant as long as there is some way of assuring continuing 
contact with the educational envi ronment . Motivation management is 
therefore defined as that which maintains student contact. Obviously, 
you have a situation which is s- boring that students will leave or 
not ,r pny attention ' 1 - learning will be diminished. One way to avoid 
this is to use contingency management procedures. In contingency 
management we make highly probable activities, such as recess, 
contingent upon the execution of low T er probability responses such 
as doing arithmetic. Contingency management is just the recognition 
that what one ought to do end what one prefer s to do at the tine 
may be two different events. Contingency management solves this 
dilemma by making the execu f ion of the prefured activity contingent 
on the emission of the required activity. Ihe instructional manage- 
ment procedure asseses some evidence of student accomplishment, such 
as the passing of a test and then initiates a high probability 
response such as an opportunity to take a ''coke" breal . Motivational 
management is often the most significant single factor in an educational 
system. 
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Let me describe how instructional management may work. We have 
a program we call the PRIME educational advancement center, We have 
organized reading and arithmetic materials into an Individualized 
program. These have been divided into steps. A task within a step 
requires approximately 20 minutes of work. Each task is followed by 
short tests which adequately measures comprehension of that material. 

Wien the student comes into our program* he first receives a 
standardized achievement tesL which is used to place him in Lite 
appropriate step curriculum his needs indicate (need management). 

His tasks are assigned, and he works them. At the completion of 
each task he takes a short test vo call a progress check. If he 
fails a progress check test he is re-routed through a remedial 
exercise (achievement nanagemen t ) . 

We initially insist that the student arrange his tasks in such 
a way that the iess preferred task precedes some higher probability 
activity. He is allowed a 7 to 10 minutes of break for each hour 
of instruction. In the back of the room we provide a variety of 
activities, such as coffee and soda pop, a chance to talk with 
friends, and card games. In other words, a menu of high probability 
activities. Eventually the student is completely free to daily 
arrange his tasks az a contract with tasks interspersed with breaks. 
He has agreed that he will not go on his break tire until he lias 
satisfactorily completed his task (motivation management), 
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It *ould be comforting to think that the engineering model 
proposed in this report would appear immediate];/ in operational 
educational systems and revive the ir ed biz n . However, the 
model proposed here is neither cacred nor sufficient. 

The present six dimensions of presentation iesu.n are a good 
first attempt to structure presentation in terms meaningful to 
the educational designer. Of course, the under 'ying philosophy 
in these dimensions and the whole model is that behavioral, technology 
is a practical solution to the needs of education. Overt considera- 
tions of behavioral technology are foreign to present educational 
practice. Buc we feel that an acknowledgement of the efficacy of 
behavioral technology and its dedicated practice in the schools 
will solve most of the. educators* problems (except for the equally 
large problem of school administration and logistics). 

We discussed presentation in terms of six empirical dimensions. 

A cont inuat it u into procedures for selecting a presentation form 
must be made. Any real situation will also be finally judged in 
the marketplace where cost, equipment transition, and completeness 
will be prime variables. Much work is needed to demise and measure 
alternative systems which will be responsive to the demands of the 
market. The creation of demand for ^ fixed product is another* 
strategy which the diversity of educators interests, goals, 
experience and their autonomy seems to preclude. 
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Media limitation is another research area which will ultimately 
produce media selection criteria together with behavioral change 
expectancy figures. 

If media preferences really exist and if learning rates of 
individuals are sensitive to media choice, then some measurable 
student attributes must be defined which will provide media 
selection guides. It would be optimal to be able to derive 
curves for expected behavioral change versus media selection 
on a real-time basis for each student, ar.d so provide material 
to achieve highest expected gain. 

There are also significant questions regarding choice of the 
dimension of presentation. Does stimulus complexity have a place 
in presentation design or should that be a concern of the generators 
of the behavioral objectives or even of the writers ana artists at 
the production end? How can the scope of c presentation form be 
adjusted to accommodate enough behavioral objectives to make a 

f 

i reasonable lesson? How can larger presentation forms be structured? 

I The game and the real-life simulation are two forms for which the 

f 

• generation of objectives must be done to fit the form rather than 

r the form depending entirely on the characteristics of objectives. 

How should the learner and content variables affect or be brought 
into presentation design? 

A better definition of timing and sequencing of instructional 
events must be offered than appears under the distribution dimension. 




- 146 - 



150 



If the objective of creating a presentation design technology 
ie to provide a med ia-indeper.dent abstract form in order to have 
a theoretical base for st rueturing learning materials and selecting 
media, how can the theory really be partitioned off from its companion 
components, media selection, behavior objective generation and 
sequencing, and even basic learning theory? If the concept of an 
abstract presentation form is to stand and prove useful, much more 
effort and thought must be put into these types of questions. It 
cannot stand or even be put into practice using only the thoughts 
presented in this paper. 

Finally, different sorts of problems must be solved in the 
practical ar, .s of teaching the yet to be developed technology of 
presentation design and sorting out the complexities of creating 
and maintaining an operational system. 

The job of educating will be only half done when a system is 
on tne store shelf. Success will be largely a function of adminis- 
tering a smoothly, operating system that works in the field, operating 
an accountable system to meet the required student goals as vas 
predicted on paper. 
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COMPUTER ASSISTED INSTRUCTION: 

AN OVERVIEW 

Walter Dick^- 
Research Associate 

CAI Center and Associate Professor of 
Educational Research 
Florida State University 

My comments on CAI are based not on an extensive amount of 
work with adults but rather on the experiences we have gained 
at Florida State working with a wide variety of students. To my 
knowledge t group at the Center in Raleigh is the only one i~ 
the country that is dealing directly with this type of application 
for adult lea rning- mate rials . Perhaps the most effective way to 
provide you with an overview of CAI, and to give you an orientation 
to our thinking about the problems associated with it, is to 
introduce some concepts of CAI and try to assess its potential 
role in adult basic education. 

The best place to begin this presentation is to define 
Computer Assisted Instruction; however this is practically an 
impossible task since it has multiple definitions which are 
related to a variety of applications. It is interesting to note 
Lhat historically CAI was defined by the J BM Corporation. IBM 
has not only dominated the field of CAI, but has in fact generated 
the label C^I. 



Now Assistant Dean for Research and Development, College of 
Education, Florida State University. 
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The original terminology for this interaction between the 
student rnd the computer was called Computer Based Instruct ion but 
after a brief upheaval in the IBM Corporaticn in the early 1960's 
the term emerged r.s Computer Assisted Ins t rue non , denoting that 
it simply was a corollary tool for instruction as opposed to the 
primary source of instruction. The field has grown beyond IBM 
at this point, however, and CAI is taking an even broader connota- 
tion. 

The best functional definition of CAI was derived by Syppes 
in terms of the three kinds of activities comprising CAI; namely, 
(1) tutorial mode of instruction. This mode clearly represents 
the type of teaching which is currently available in program 
instruction; that it., the content area is systematically analyzed 
ar.d performance objectives a::e developed. The student receives all 
his instruction in a particular topic vi 1 an interactive dialogue 
at a computer terminal. This type of instruction is typical of 
the efforts at Pennsylvania State, Texas, University of Illinois 
and to a certain extent Florida State. (2) The drill and practice 
type of activity is clearly best represented by Syppes activities 
at Stanford. He has provided math and spelling materials to 
students by means cf teletype terminals which are connected via 
telephone lines to a PDP/10 system. He has achieved from good to 
outstandir.g results by using the computer primarily as a presen- 
tation device. The drill and practice mater'als contain informa- 
tion which the student has previously encountered in the classroom 
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but in which he needs continual practice, immediate feedback, and 
repeated presentations of difficult material in order that he can 
achieve these basic skills. (3) The third area within the definition 
of CAT activities is that of problem solving. Problem solving is 
generally most like what we call simulation or gaming. Gaming or 
simulation is often used when a student already has the basic infor- 
mation about a topic, but he is required to utilize th^s information 
in his interaction with the computer in order to derive a deeper 
understanding of the concepts which lie has learned. Kconomic games, 
business games, and medical diagnosis simulations are all typical 
of CAI. Games and simulations are often very difficult to program 
in this area and have probably been lc^st pursued in terms of their 
learning potentiality for the student- 

Since the tutorial mode of instruction is perhaps the one in 
which is mo. c ; familiar and prevalent in CAI, it is worth taking 
time to discuss in some depth the Florida State CAI Physics Project, 

I will not argue that our developmental techniques, or necessarily 
our outcome, are typical of the general use of tutorial systems 
throughout the country; but I think there are a number of generaliza- 
tions which we have evolved from this project which may have a 
great d^al of bearing on now you might consider the use of the 
computer i*i aduit basic education. 

The computer based multi-media physics course was funded by 
the U. S. Office of Education in order to provide us with support 
to explore the possibility of developing and evaluating an automated 
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college physics course. Approximately 300 to 700 students enroll 
in the physics course each quarter and it was from this group that 
we obtained our students. 

When the project began, we undertook two activities which 
proved to have high pay-offs; first, we video taped the lecture 
presentations of the professor who is responsible for the class- 
room version of the course. We utilized the tape to do an 
analysis of the content of the course and to determine what infor- 
mation was being presented to the students. These tapes proved to 
be invaluable to CAI writers at a later date. 

In addition, in order to show some immediate activity on our 
computer system, we secured copies of the course tests, analyzed 
the types of skills that were required in order to solve the questions, 
and analyzed the homework t.iat students were required to do. We set 
up a series of home work review problems which were made available on 
an optional basis to the students in the regular physics course. 

The students were offered the opportunity to come to the Center and 
receive homework test-like questions which \cere similar to those 
they would be receiving on their conventional test in the physics 
course. The computer supplied answers to the questions as well as 
reredial feedback and reinforcement. 

You can imagine what it was like the first time we entered 
the class of 700 students and offered this service to them. We 
had to explain in three minutes what CM w’as all about, and why 
they night want to come down and participate. Out of about 700 
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students only 30 volunteered to participate. The second time there 



Were 75 to 80 who came down; and by final exam time, the Center and 
its terminals were saturated for nearly a week prior to that exam, 
processing approximately 280 students. This was tremendously 
encouraging to us in that the growing number of students indicated 
that in their minds we were providing a service that was of some 
importance to them. A great deal of data showed us where the problem 
areas were. We were able to identify those concepts in physics 
where they were really having difficulty. We used the video tapes 
and this student base line information to go about the development 
of the individual lesson sequence. 

I think that it is important to note from these early activities 
that, in general, it is unwise to develop a C'AI course, by first 
programming lesson one, then lesson two , then lesson three, and so 
on without first bringing these materials before your students, 
getting their reactions, performance data, and some kind of base 
line information about the current level of capability, before 
you begin the actual detailed programming. A gr .t deal of time 
is v T asted in preparing instructional sequences with extensive 
remedial loops which are eventually of no value to a significant 
number of students. 

An important decision Was to make the CAI physics course 
a multi-media course. We made an early decision not to depend 
solely on the computer as a presentation device but to take 
advantage of the excellent films which have been dove] oped by the 
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Physical Science Study Committee. In addition, ve took advantage of 
the film skills of a ptoiessor at Florida State who produced film 
loops of the experin ent s usually carried out by the instructor in 
front of the class. We also developed audio tapes which were, in 
essence, compressed versions of some of the lecture presentations. 

And we utilized, of course, a text bock which the students were 
required to read prior to starting a lesson. 

We used the CAI system itself for monitoring and guiding the 
students throughout the lessons, for continuallv reviewing ♦'heir 
progress in both the testing and review mode, and for providing 
remedial information to the students. This approach to the 
development of the CAI course optimized the use of the resources 
which were available to us. Although there are no clear cut 
rules at this point which say that this should be presented via 
computer and this should not, it is our opinion that it is inefficient 
to try to do everything on the computer. 

The third important aspect which evolved quite early in the 
project was that of staff differentiation. I think this is a 
critical concept in the development of CAI materials. We utilized 
several professors from the Physics Department at Florida State to 
provide the overall conceptual leadership in the development of the 
instructional materials. This was a role which they plaved very 
wellj however, it would have been foolish and naive of us to 
expect them to spend a great deal of their tire in actually writing 
and debugging all the instructional materials. These men are busy 
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with important research projects, and contrary to some of the early 
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IBM thinking, there is no payoff at the present time for a professor 
to develop CAI materials. Therefore, we hired three physics writers, 
one with a master T s degree and two with bachelor T s degrees in physics 
who were competent and who were interested in the instructional 
process. They translated the general conceptual view of the 
physics professors into meaningful, instructional materials. 

In this process, they interacted with the psychologists who 
were available in our Center and to people who were completely 
informed as to the capabilities of the computer system itself. 

These writers were the backbone of the actual development of the 
instructional material which was implemented on the computer as 
well as on other media. When the materials were completed they 
were h. led to the coders. These are people who are expert in 
the CAI language utilized by our computer; they did the first 
debugging of the materials and actually entered the information. 

We had computers operators, also, who ran the CAI system, computer 
programmers, who developed a data analysis system for us, and 
proctors wTiose role it was to work with the students as they came 
to the Center. These people, along with secretariat help, characterized 
the type of staffing wl ich ve utilized in the project, I cannot 
emphasize too heavily the necessity of having such role differentia- 
tion, and for maximizing the utilization of people's capabilities 
where they can contribute most significantly without dissipating 
their time and energies across a wide range of tasks which would 
take much longer to master. 
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During 1968 and 1969, vc conducted three field studies in 



order to evaluate the multi-media computer based physics course. 

Our students were sampled on a volunteer basis from the students 
regularly enrolling in the Physics 107 course at Florida State. 

The administration granted the CAI students the regular three 
credits for this course with the stipulation that the students 
take the regular exams which all the other students in the 
course would be taking. 

In the first study we had 23 students who took their entire 
physics instruction by CAI. The outcome was a significant difference 
in the final grades of the students who took the course by CAI 
compared with a matched sample of students from the regular course. 
There w f as also a considerable time savings; however, we did not 
witness any great acceleration of ti.e students 1 pace through 
the materials. In general, the students came to the center to 
take two or more lessens a one time and primarily scheduled their 
time in the Center when it was convenient in relation to their 
other course responsibilities. 

The second study was conducted after the CAI materials vere 
revised. We achieved in this study approximately equal performance 
between the CAI group of 37 students and a matched group of students 
in the regular course. Again, there was approximately a 13 percent 
saving in time. 

For the third field study, we removed the films and most of 
the audio tapes from the course presentation so that the course was 
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only two thirds the length it had been in the past. The other third 
of the course time for CAI students was spent in class discussions 
of topics related to the physics course, and extension of these ideas 
in more advanced areas. The results of this study indicated that 
those students who completed all 29 CAI lessons performed equally 
veil on the final exam as those students in the conventional course. 
The outcomes of these studies make a number of things quite clear 
to us; for example, the students see the major benefits of CAI in 
terms of its self-pacing aspects. There is also a great sense of 
working along with a tutor. 

As a part of this physics project, we attempted to evaluate 
the cost/benefit of CAI. Immediately ve encountered a problem in 
demonstrating the value in a monetary sense, of enhancing learning. 

In the University system budgets are generated in terms of credit 
hours produced and not grade points; and therefore, it is difficult 
to place a monetary value on an "A" as opposed to a "B” or to 
determine the value of achieving ten objectives as opposed to five. 

However, in the final analysis, trends seem to indicate that 
the cost of CAI will continue tc» go down, and hopefully the 
effectiveness of CAI will increase within the next decade. It is 
apparent that the homework problems that were utilized by the initial 
students in Physics Project, prior to their various examinations, 
contributed approximately 10 percent gain3 in their learning outcomes 
on the exams. This appears to be a potent and very rewarding tooL to 
students and ve are considering the application of this particular 
aspect of CAI in other areas. 
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In summary, we have discovered and evolved a number of generaliza- 
tions about CAI curriculum development that you may wish to consider 
when other people talk to you about their views on this topic. The 
first of these generalizations deals with the systems approach. We 
evolved and developed our own veision of the systems approach and 
we progressed through this project. 

We did in fact analyze our tasks and our context. We did get 
to know the entry behavior of our students; we did formulate our 
behavioral objectives primarily on the basis of behaviors expected 
of the students on the tests. Instructional strategy was one of 
utilizing a variety of instructional media in order to try to 
optimize the capability which was represented by each medium. We 
evaluated as often and in as positive a fashion as we could. 

You can learn the jargon of the systems approach in a relative 
short period of time; on the other hand, it is a very difficult and 
complex thin r 3 to seriously follow all the guidelines and procedures 
which such a model suggests. However, we feel that tht curriculum 
development efforts of CAI are worth this effort and there is 
beginning to evolve some evidence that in fact this effort is worth- 
while. 

The second generalization is that the higher the criterion 
performance which you expect of the students for whom you are 
preparing materials, the more difficult will be your task of 
programming the course, and the more complex will be your 
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instructional strategy which you will have to employ, thereby 
complicating the 'development of your instructional materials. 

Third, we are convinced that CA1 is a tool which should fit 
within a curriculum and within an Instructional setting; that is; 
it should not carry the entire instructional load but should be a 
part of a multi-media resource. One thing to keep in mind, however, 
is that the greater the variety of media which you choose to employ, 
the more complex will become the logistics for the student, and 
the more difficult will be the total implementation process. 

Last, let me emphasize a point ir.aae earlier; you should be 
constantly aware of the role of staff differentiation, which 
seems to hcve high pay-off in terms of carrying forward a curriculum 
project. In other words, the burden should be spread among a number 
of people who have special aptitudes or training for tha various 
tasks. Specialization within a CAI project undoubtedly increases 
your probability of success. 

The Hardware 

I would like to talk briefly about the ocher side of the 
coin in CAI, namely, the hardware or comp ter aspects of it. 

To educators, the computer itself is probably the least interesting 
and certainly the most costly aspect of computer-assisted instruc- 
tion. In principle many of the computers available tod; y from 
numerous manufacturers can be utilized for computer-assisted 
instruction. Ihe primary reason why more different manufacturers' 
computers are not being used for CAI is the lack of CAI operating 
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systems. By this I mean that the software required to assemble the 
course material and to provide it in a time sharing mode with a 
number of student terminals is not available, and it is costly to 
develop . 

The CAI computer system that is predominately found in the 
United States at the present time is the IBM/1500 system. The 
currently available versions of the 1500 system have a 1130 central 
processing unit which rents for less than $2,000 per month. How- 
ever, the peripheral equipment require., to complete the CAl system 
brings the total monthly cost anywhere from $8,000 to $12,000 
depending upon the variety of audio-visual devices added. Clearly 
this cost is prohibitive in an operational setting. In nearly 
every CAI Center, the federal or state government is paying the 
bills in order to provide researchers with the opportunity tc 
explore the potentials of CAI. 

There are at least two solutions to this problem of CAI 
equipment cost. Perhaps your first inclination is to think 
of adding many terminals to the computer system in order to drive 
down the hourly cost per student. This concept has received sore 
attention. The IBM Corporation is now making Coursewrit^; II 
available on its 360/40 computer system. As such it should be 
capable of driving more than 32 terminals; however the cost of 
the terminal devices is still prohibit ive . 

Several feasibility studies have been completed for the l r . S. 
Office of Education on CAI systems which would nerve 1000,000 
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students in a metropolitan a ■ oa u'uJ would have appro:-: mately 1,000 
terminals . lhe University or Illinois is currently attempting to 
develop a C l computer system which wo-Jd drive 4,000 terminals. 

I have no doubt that in time these s stems will become operational; 
however, one of the primary costs involved in such large systems 
is no longer the cential processing unit but the telephone line 
charges because most of these systems require one telephone 
line per terminal. 

The second cost is the terminal devil a itself. Cathode 
ray tube (CRT) terminals are much like a TV set with a typewriter 
key board Ihese terminal devices sell for over $j,Q00<. For 
computer systems with, hundreds and thousands of terminals the 
cost would become extremely high- However, the University of 
Illinois is working on a low cost plasma screen terminal which may 
become operational in the m j ar future. This terninal device 
should overcome some ol the current technological problems and 
costs associated with the CRT. 

The other development, in terms of the iuture of C4I hardware, 
is one that few 1 people are espousing but at liorida State we feel 
that it has distinct possibilities that is providing CAI on a 
small computer which is a s: and-. a loiu , lotallv controlled, reason- 
able priced computer system. ihe results oi cm analysis indicated 
such systems could he made available f a approximately SI'S, 000 to 
$185,000 or at a rate <»f approximately Sib. 00 to S20.U0 per student 
per year. This system, vo* id i iv lode * CAI operating ^ystvn and 
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16 to 32 student terminals. The cost for this proposed system would 
be approximately 1/3 of the cost of the present 32 terminal CAI 
systems . 

We also speculate in this regard chat as time goes on two eco- 
nomic hypotheses will be com* reality. First, although the decrease 
in the cost of che computer central processing un will begin 
leveling off soon, we feel that the cost of the peripheral equipment 
and the terminal devices will continue to go down. Likewise, it 
seems a fact of life that inflation will continue to drive up the 
cost of instructors' salaries - that fixed item which represents 
anywhere from 70 to dO percent of the educators' budget. These 
two trends argue for the enhanced economic feasibility of CAI. 

* 7 ith regard to computer systems, I would like to describe 
briefly one study which we have done at Florida State University 
which I think may have particular relevance to the field of adult 
education. This study began as a rather mundance comparison of 
student performance on a programmed text with performance by 
another group of students on a similar program presented by 
CAT. Although this is not a very startling idea there is remarkably 
little evidence of this type of comparison being carried out, at 
least it is not being reported in the literature. 

This project evolved from a graduate student's efforts in 
which he developed a PI text which was utilized as source material 
for the development of an enhanced CAI course package. The package 
consisted of about 40 minutes of instruction on the topic o f 
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significant numbers, a topic which is usually taught to eighth 
graders. The study was run once and the data were analyzed and 
replicated under better experimental conditions. 

The primary finding of the study vas that with high ability 
eighth grade students there was essentially no difference in their 
performance i r * of being on CAI or using PI: however, the 

performance on the final exam of the low aoility students who used 
CAI vas significantly less than that of the lev: ability students 
who utilized the PI text. As a matter of fact, on a retention 

test the low ability student? using PI actually scored better 

than the high ability students using n I The important point 
here is ♦'hat the device that the low ability students were 
using in CAI treatment was a cathede ray tube terminal. The 
program did not provide the students with an opportunity to review 
in the sense that the student could review a PI text by flipping 
back pages. Information appeared on the screen, the student 
responded to it, and it vas gone. The results seem to indicate 

that the low ability students may have greater need for memory aids 

or actual hard copy printouts. Perhaps this result might be 
generalized to include adults who night use cathode ray tube 
terminals . 

On the final note on the utilization of a variety of instruc- 
tional devices that are driven by a computer. One point of view 
would argue that in order to maximize learning you must have all 
the instructional devices under the control of the computer*, ;h*.t 
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is, we should have a random accuss audio device, cathode ray tube, 
light pen, key board, projector, and even TV and other display 
devices under computer control. It has been our feeling, based 
on some experience, that we prefer to utilize only the basic CKT 
S' key board, and light pen capability on the CAI system. This 

decision has been reached on the basis of, (1) the cost of additional 
\ devices, and (2) a genuine questioning of the trade-off or benefit 

| which is to be derived from additional devices. We have found, for 

example, that students as young as junior high age are quite 

1 

capable of utilizing a Kodax carrousel projector appropriately. 

| We have also utilized flip pads which contain graphics, and vork- 

| books for additional writing activities. None of these devices 

i 

\ 

{ was under computer control. We do not feel that the student has 

been handicapped in any way because of the situation. 

In summary, the intellectual and economic battle, will 
continue over the next several years as desirability of large 
computer networks for CAI is compared to small, stand-alone, 
full type purpose computers. The functional characteristics for 
the terminal will also become more important. If you are fortunate 
enough to have the funding available to actually implement CAI , 
there will be a number of pitfalls which should be avoided, or at 
least prepared for. 

Let me briefly indicate the two primary areas of concern 
which 1 have identified. The first occurs in dealing with computer 
manufacturers. There are many problems which you can anticipate 
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in terms of delivery dates, total cost, one time charges, maintenance 
scheduling, and the like. I don't mean to imply that the manufac- 
turers are out to deliberately deceive you; but often, if you are 
in a city in which they have never heard of CA1 until you order 
your CA1 system, it is very difficult to get people from within 
a manufacturer's sale organization who are familiar with CAI. 

Second is the problem of continuous on-the-job training of new 
people. We have found that after establishing a staff of 
computer programmers, authors, operators, coders, and psychologists, 
we have been able to increase their effectiveness by various 
on-the-job training techniques, for it is almost impossible to 
hire the services of trained CAI people. Florida State is one of 
the few institutions in the country where there are specially 
trained people with capabilities in the area of CAI a.id we are now 
producing our first significant number of doctoral candidates. 
Computer-Man a ged Instruct! on (CMI) 

As a final portion of my presentation, I would like to discuss 
some alternate applications of CAI and mention a sample of the 
research studies ve have conducted at Florida State, pointing out 
what 1 consider possible implications in the area o! adult elucation. 
Let us return to the classification scheme, namely drill and practice, 
tutorial, and problem solving. Although I spent sore tire discussing 
the tutorial mode, I would argue that the drill and practice r.jde 
will also have considerable application in areas of critically 
important learning skills in adult basic education. 1 he very nature 
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of a CA.I system makes it a very private framing situation in wiiicli 
the student can make numerous mistakes a, id til; computer will continue 
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to provide him with the type o t material whic 1 he needs in order to 
master certain basic skills, 'lhe sane situation repeated in the 
classroom would be embarrassing and humiliating- 

Perhaps the most immediate demand £.,*t co puters in v due at ion 
is what is now being called computer managed instruction or CM I. 

If you consider our physics project in which we used a variety of 
media, the computer played primarily a ocrh'oiim; and an evaluative 
role. It is my understanding that inis is what the majority of 
professionals would <lesi rile as oongut c r -maivige.d instruction. 

CMI is the overall management ui learning matt rials and evaluation 
of students who are par l icipatiug in a training program where the 
instruction is nut primarily conducted by the computer but by the 
use of various types of instructional mat. rials 

Project plan of the Ar.ieri.can Institutes of Research is one 
example of CMI and Bob Glaser’s work at the University of Pittsburgh 
with individually prescribed instruction, which new includes the 
implementation of a computerized monitoring system, is another. It 
is my observation that the type ol Jnstructio 1 in which objectives 
are set for the student, and materials are pi.*ci<ie<2 to achieve those 
objectives, is becoming more evident. A stud nt rviy select, those 
objectives in a sequence .. deli he feels is ce >t for him, oi the 
sequence nay be prescribed by the corput»r. ' U- tudent is totally, 
or partially, evaluated b‘ the .omyJt*r on 1ns acquisition of those 




objectives. It is a very significant step forward, not simply in 
the utilization of computers in education but in our philosophy 
of education per se. 



In order to give you a concrete example of computer managed 
instruction, I would like Lo describe the graduate level course 
which is conducted via CMI at Florida State. The name of the 
course is Techniques of Programmed Instruction; its purpose is 
to have students produce between 30 to CO minutes of programmed 
in s t r uc t ional materials which are developed in accordance with 
a systems approach model. Of course, under computer managed 
instruction, the content ha:> been task analyzed and 20 primary 
tasks have been identified. For each task there are from 
one to three objectives. One half of the students are given 
a set sequence of objectives which they must follow; the other 
half are able to select any objective or task which they feel is 
appropriate . 

After choosing an objective, the student is given a list of 
the resource materials which are available in order to achieve 
the objective. For each task there is basically, 1} a cognitive 
skill, such as proficiency in identifying the components of a 
behavioral objective, and 2} a product such as writing behavioral 
objectives. Ultimately, through reading the materials, meeting 
in small groups, and requesting presentations as they see fit, the 
students proceed to develop evaluate, revise and again evaluate 
their instructional materials and document this whole procedure in a 
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final report. The primary role of the computer is to evaluate the 
students' performance on each of the cognitive tasks and to maintain 
a continuous record of where the students are in the course. 

Student requests for assistance can be raised with the graduate 
assistant, or the professor, We are analyzing both the role the 
professor plays and the ability of the computer to manage this 
relatively large graduate course. 

As far as I know, this is the only application of Computer 
Managed Instruction in which students actually interact with 
the computer via a terminal which is operating in real time or 
in a CAI mode. In other applications of computer managed 
instruction, the student, when be has completed his instruc- 
tion on an objective, takes a test which is then scored by the 
computer. The report is sent back either to the student or to 
the teacher who determines the next instructional sequence. 

Other Applications . 

The CMI course which w’e have implemented is a rather small 
beginning effort to try to understand the variables and complexities 
associated with computer-managed instruction, A larger effort which 
may be more prototypic of total adult education training programs 
is one that we are mounting in the area of elementary teacher 
training. Florida State has been awarded a U. S. Office of Educa- 
tion contract to develop the second phase of a model teacher training 
program for the preparation of elementary teachers. The Florida 
State conception of this training relies heavily on individualized 
instruction. 
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The present enrollment in this program at Florida State is 
approximately 800 to 900 students, with projections that can go as 
high as 1500 students. In order to implement an individualized 
program of this magnitude it is imperative that the computer be 
involved as a monitoring, scheduling, and evaluative device. 

The computer will continuously print reports for counseling by 
clinical professors in order that they may spend their time 
advising students in terms of the use of resources. The details 
of this model have not yet been worked out but it is a challenging 
task ar.d one in which the computer will be an invaluable tool. 

As you can sec, by including these other examples of CAI, 

I am trying to broaden the concept of CAI to include more than 
just a tutorial mode of instruction. For exanple, we have run 
several studies In which it is quite clear that using a branching, 
testing strategy based on test item characteristics, one can very 
quickly and efficiently test students in a series of concepts 
without administering large batteries of questions. The final 
lev’ll of performance can be quickly identified. We have used the 
term sequential t> sting for describing the process in which a 
student is provided test iiems which approximate his level of 
performance on the preceding item on th° t^st. The test is 
specifically adapted to his perform in v : » .e proceeds through 
it; therefore, this type of testing necivity can be utilized x . 
the same way conventional test are currently used- However, it 
is much shorter and quicker and has been shown to provide more 
reliable data, 
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Another interesting example of using CAT in a testing mode is 
currently under way at Florida Stated Department of Social Welfare. 
Again, this may have same application to adult education. The 
students who typically enter the Graduate Department of Social 
Welfare come from a great variety of backgrounds. The initial 
graduate courses are often trivial to some and over the heads of 
o the rs . 

A battery of test items is being established in the area of 
psychology and sociology which will be made available by C'AI to 
students entering the Social Welfare program. They will take these 
tests and be systematically evaluated at various concepts- For 
certain levels of misunderstanding, remedial activity will take 
place at the terminal. For students with very poor performance, 
additional materials will be made available and they will return 
and be tested later. It is anticipated that in this manner students 
can be screened and placed in either remedial activities or regular 
courses appropriate to their interests, skills, and program. They 
will no longer be required to participate in courses which are 
either beyond their capabilities at the present time or are rela- 
tively trivial on the basis of their background or undergraduate 
experiences . 

Another successful utilization of CAI has been in the area of 
academic advising, A study of this application was made at lie 
Tallahassee Junior College by one of our graduate students. He 
placed two terminals at the college and programmed a sequence of 
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information which was felt to be of value to the students. This 
information included the rules and regulations of the Junior College, 
information about grading standards, information cn the basic 
studies program required, and courses that were available within 
that basic studies program. There was also information available on 
transferring to various colleges in the southeast, as well as 
specific information on Florida State University. The purpose 
of the activity was to provide the student with information which 
he needed to better understand the Junior College and its rules 
and regulations, and to give him official information about the 
program. The computer also provided the student with i listing 
of his current record at the college in terms of the courses he 
had taken, grades he had received, and his grade point average. 

On the basis of his information about the junior college 
curriculum and his record there, the student at the end of his 
interaction with the terminal was requested to list the courses 
which he would like to take during the next academic quarter. 

The information was stored in the computer and evaluated by the 
counseling team at the college. Approximately 85 percent of theu 
records generated by the students were judged to be good or 
excellent by a faculty jury. The system was heartily accepted 
by both students and faculty members. It is interesting to not t 
that this is in a junior college where they take pride in the t\: 
of counseling they can provide the student. This application is 
cal of the use of computers to provide accurate information for « 



diverse population. 
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A Significant Question. 

I would like to conclude this presentation by raising a question 
that is often debated by CAI personnel; that is, should CAI be used 
primarily as a tool in which we implement programmed instruction 
material or is it a totally different concept ionali zat ion of the 
problems and capabilities which we can utilize? In other words, 
should ve use CAI basically as » ^age-turner , or should we make 
it a very esoteric device? I think my answer at the present time 
would be to straddle -he fence with one loot on either side. X 
would argue that if we are to prepare meaningful materials, get 
them running on a system, and begin producing results, the best 
way is to start with programmed instruction materials which 
have been developed with a leasonably good systematic approach, 
and utilize these materials to become more familiar with the computer, 
and to learn about our students' reactions to it. Unlike other 
media, the computer has an extensive memory system; it can make 
decisions and it can control media devices other than itself. It 
is because of these capabilities that many people have become 
enthusiastic about the long-range capabilities of CAI, Within 
the past year, we have begun exploring some of these capabilities, 
hopefully in a realistically simplistic systematic way to try to 
determine whether we can manage these capabilities. 
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EVALUATING PROGRAMMED INSTRUCTIONAL MATERIALS 



Ronald H. Sherron 
Director 

Adult Basic Education Evaluation Project 
Virginia Commonwealth University 



We have heard a lot about programmed instructional materials 
and how to develop then, but we have heard very little about 
evaluation; yet th is is one of the most important aspects of 
instruction. Evaluation is an appraisal, and appraisal calls 
for judgment Judgments ere based on a critical consideration 
of the bes * evidence available, with evidence being the key word. 
We need evidence if we ara to make judgments and evaluations. 

Now, how is evidence collected? In man’s search for truth 
he has traditionally used several approaches. Let’s take a look 
at four or five of them in some detail. (1) One is tenacity. 

We hold to ideas we have always believed to be true; they are 
handed down from generation to generation, and sometimes ve cling 
to these truths even when i ' aie is evidence to th? contrary. We 
all know many examples, such as the early belief that the earth 
was flat. (2) Another is appeal to authority. We accept the 
ideas of subject matter experts, for example, because of their 
training and experience, but unless they ’eep abreast of new 
discoveries their advice may be conflicting. (3) Intuition is 
a third approach, but you know how often a hunch fails to pay 
off. (4) An additional approacu is reason. There is t'\e belief 
that ve humans can arrive at truth Lv rational thought. Often, 
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however, such reasoning is quite false. A person can be very 
logical, but if the basic assumption tnat he starts with is 
incorrect, then his conclusions will be incorrect. (5) Finally, 
there is the scientific approach, using data that can be verified. 
There is testing involved, which is open to inspection, and you 
draw your conclusions from the findings. You don’t just say some- 
thing is true because you believe it is true; you say it r s true 
because of evidence you offer. Objectivity is always a problem; 
it is very hard to be objective, especially when the data doesn’t 
support your assumptions. 

There are several of the methods man uses to search for 
truth or facts. Scientific thinking and scientific methods are 
a combination of various approaches. They may be labeled inside and 
outside. The inside methods are reason, formal logic, the deductive 
method, rationalism; this is the ivory tower. The "outside" methods 
are experience, observation inductive thinking, probability, criti- 
cal analysis. These methods, deductive and inductive, are both 
used in the scientific method. The scientific method is character- 
ized by, or is _ased on objective data. It uses hypotheses to test 
data: it is analytical and it strives for objectivity. It develops 
tools of measurement. We look for specific evidence and we use 
statistical techniques to compare findings. Measurements are 
developed and tested to assure validity and accuracy. 
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Evaluation Employees Scientific Thin king, 



With this introduction then, ve will discuss evaluation as a 
type of scientific thinking. The whole notion supporting research 
is the fact that you can define it in some understandable form and 
it can be verified. It promotes, se If -improvement , and self-study 
for a person or an institution. The end purpose of evaluation 
is to improve what you are doing, to see if you have reached your 
objectives; and if you haven’t, then to change things or try 
another approach. Evaluation involves three basic steps: (1) stating 

in precise terms the objective; (2) securing evidence relevant to 
t:he objective; then (3) drawing your conclusions or making a judgment. 
In other words, you collect evidence, you review evidence, and then 
you make a judgment in regard to the social utility of the process 
or activity. 

Start With Objec t ives . 

There are many approaches to evaluation. If you want to 
evaluate a method, or a program, yot must know the goals and 
objectives of that method or program. Then you secure evidence 
to find out whether the objective has been accomplished, and 
you relate the evidence to the objective in various ways through 
statistical computations, and through rational judgment. We 
use all of our various means and their validity varies with the 
method We are using. The more control we have over a situation 
the mole likelihood that our findings are valid. 
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Let's take a little closer look at analysis of objectives. 
First of all, objectives must be stated in terms that allow 
appropriate evidence to be collected. If I say one of my 
objectives is to teach you to understand Shakespeare, what do 
I mean by understand? How will I know that you understand 
Shakespeare? We define understanding in behavioral terms, and 
these must be observable behaviors. Sometimes it is hard to 
know what is going on in a person's mind and we only see secondary 
evidence of learning in terms of psycho-motor performance, 
cognitive behavior, or affective behavior. 

Ways to Collect Evidence . 

How do we secure such evidence? They involve both subjective 
data, such as opinions and attitudes, and objective data such as 
test scores; but all data collected must be clearly related to the 
objectives. Observation is perfectly legitimate. It is one of 
our oldest and best methods. Questionnaires, interviews; and 
analysis of cumulative records; all of these are legitimate 
methods for evaluating. The basic question is: to what extent 

do the available data indicate that the desired objectives have 
been, or are being, effectively accomplished? This is really 
whet we are trying to find out when we evaluate programmed 
instruction materials, whether we have effectively accomplished 
what we are trying to do, the key v ' rd being "effectively". 
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C rite ria By Which To Eva lu a te Mater ials . 



Now how do we know whether we are effectively accomplishing 
this objective? We need some criteria. Everybody has criteria; 
for instance, you have your criteria for a good meal, and I have 
mire. You have your criteria for an educational program, and I 
have mine. One reason we need criteria is that thousands, even 
millions, of dollars are being spent every day on educational 
systems, hardware, and software. There is no end to the variety, 
and the manufacturers are turning out materia _s and equipment by 
the tons. Adult educators are buying these products, some of which 
are effective, and some are not worth the paper it r s printed on. 

We need to study and find out which systems, and materials will 
accomplish specific learning objectives with persons having 
specified learner characteristics when used in a particular 
instructional environment. 

Educators have been told that PI materials are created by 
testing and revision, and you expect evidence of performance. 
Evidence of performance is often poor, or is not supplied. There 
Is a growing concern that PI materials are not being adequately 
field-tested. If you will review a program manual, you will see 
the kind of field testing done. For example, program producers 
will say that they tried the program out with 80 Uka Indians and 
put it on the market. Or they will say that it was tested in the 
Job Corps on 2000 people, but all of one type. How representative 



177 



o 



ERIC 




are the materials? Will they work in your unique situation with 
your people in your geographic location? 

Three professional organizations cooperatively drew up criteria 
for evaluating programmed materials, The NEA formed a joint committee 
involving representatives of the American Educational Research 
Association, thi American Psychological Association, and the Depart- 
ment of Audio Visual Instruction, The committee published various 
reports. 

They recommended criteria in two categories, internal character- 
istics and external features, or internal and external criteria. 

The Internal characteristics or features can be revealed by visual 
inspection. From inspection some of the following basic character- 
istics can be assessed: (1) Detailed effective tasks to be 

accomplished with the content organized in such a way that you 
can jne vhat the objectives are, (2) Level, forms and style, 

(3) The types of programming. (4) The length of the frames that 
are in the PI book. In addition, in editor's check list will 
note these essential items: (1) organization, how is the book 
organized and how the system is organized, (2) tie difficulty, 

(3) accompanying checks, (4) accessory materials — their conven- 
ience, ease in use, (3) the physical features, and (h) the cost. 

External criteria are features that are not observable nerely 
by inspection. These call for a little more detail and study: 

(1) What was the source of the program content; (2) what were th ■' 
qualifications of the author; (3) what about the history of the 
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program, how was it developed; (4) is there an explanation of the 
tryout and revision that it went through, if any; and was it field 
tested; (5) are there any test data, are -here data about the 
characteristics of the students used for validation? Was this 
population similar to the people that you want to use the programs 
with? If not, it is probably not appropriate. 

There are critical reviews, you can read your professional 
journals, you can read your audio-visual magazines, there are 
various ways to gain knowledge of the program. Research is being 
completed and reported in education and related disciplines. Read 
these reviews and reports before you buy a program. 

Eva l uatio n of Results . 

Most important to evaluate are the products of rhe instruction. 
These can be measured through achievement, retention., or transfer. 
Information is needed about instructional effects. How much did 
the student achieve? Was there any retention after he closed the 
book , or is that the end of it? Will this undereducated person 
be able to go out and make any application of learning? Can he 
make the change in his life style, in his self-concept as a result 
of what he has learned? These are important questions to which you 
should address your attention. Very few program materials on the 
market answer these questions, oo you may have to conduct your 
own evaluation studies. 

Here are three basic considerations evaluating programs: 

(1) Evaluate each program according to its merit in producing 
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the specific outcomes that you want, your objectives. If you 
don't know what you want the program to do, how are you going to 
buy a program to accomplish this? You must have objectives for 
your program, and then you seek the materials that appear likely 
to achieve them, or you can collect evidence to support your 
assumption that it will accomplish your purpose. 

In other words, you ask of a given program, is it appro- 
priate for your purpose? Is it practical, and do you have evidence 
of the effect that this program will produce? The Guide To 
E val ua t ing Self-Instructional Pr og rams tells you how to set up a 
small study to do your own evaluation. After screening some of 
these hundreds of programs that are on the market, you may narrow 
selection down to two or three. Then if you are not sure which to 
choose, you can run a small pilot study. 

(2) There are criteria for validating your own materials; 
some recommended by the same three professional organizations 
I mentioned earlier. The evidence should be based on a carefully 
conducted study which shows what the programs accomplish. People 
should report these validation studies in educational journals or 
in project evaluation reports. If the adult education centers 
will share those, and if the Adult Education Clearing-house in 
Syracuse will report studies in formats according to these criteria, 
it would eliminate a lot of our problems because Just about every- 
thing on the market is already being used by som< Vdy in the field. 
It is a matter of finding out whether a program is effective. Most 
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people haven't bothered to evaluate v;hat they are using. They are 
just using the materials because of convenience or other subjective 
reasons few are happy and many are dissatisfied. Host users really 
don't know what their programs are accomplishing. Results should 
be carefully documented. If you are the director of a program and 
you say your program works, you ought to be able to substantiate 
this in some objective manner, not rely only on your own opinion. 

An opinion stated by an expert is fine; but all claims to effec- 
tiveness should be supported by specific evidence. 

(3) The last thing we need to look at, a very practical matter, 
is cost. This may be the most important criterion. All the others 
are important; but sometimes this is the one on which you really 
make the final decision. You may find an ideal program but can't 
afford it. You figure what the program is going to cost you per 
unit, per student, the equipment cost. Some software doesn't 
require equipment and others do. What is your initial investment 
going to be, can you afford it, what is the long term investment, 
what is the training time per student and teacher needed in order 
to use the system? How long does it take the system to accomplish 
what you want to accomplish? What quality of student is required 
by this particular program; in other words, what student character- 
istics and instructional setting is this program designed for? 

You need to know the answer to these additional questions. 

What kind of instructor will be required, or nonitcr, or proctor? 
fan a student aide administer it, or will it take a professional? 



‘ are logistics , space Fr qun ements ' i r ou only have so much 

j 

\ space, where or how do you store tne equipment? Ihen there are the 

i power and maintenance requirements' What is the useful lii;e of 

| the hardware and software? Some pregrams a:e designed to run out 

t quickly, what will it cost. r . o t.oJc r ivizc them? In?}e are all 

* 

* important consider ations m evaluating progr^r: instruction 

| . materials. 

J In conclusion, the responsibility for demanding quality 

■ instructional systems resides .-:i th the pro:, e s sional educator, 

| who must become aware of, and he able to articulate program 

J objectives and acceptable evjdenc*. u I learning. 
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INTEGRATING PROGRAMMED INSTRUCTIONAL 
MATERIALS INTO ABE PROGRAMS: 



THE LEARNING LABORATORY APPROACH 
Joseph B. Carter 

Coordinator of Learning Laboratories 
North Caroli a Department of Community Colleges 
Raleigh 




In 1963 North Carolina educators and politicians recognized 
the urgent necessity for an educational program that would meet 
the need of every adult that was underemployed, unemployed and/or 
desirous of more educational training. In the same year the N. C. 
Department of Community Colleges was established. During the past 
six years, 39 technical institutes and 13 community colleges 
have been established and are now in operation. By establishing 
these institutions it was possible for the state to extend universal 
education and opportunities to every adult, regardless of his previous 
educational experience or training. The community colleges and 
technical institutes offer programs in college preparatory work, 
vocational and technical training, general adult education, basic 
adult education, and community services. 

The Open Door Policy was adopted and has h r« d its gieatest 
impact on basic and general adult education programs. The Open 
Door Policy states that any adult will be accepted into the program 
and given instruction relative to his abilities, interests, and 
goals. We knew that any instructional program that ve could 
develop would have to be individualized, so started out in the 

fall of 1933 on an experimental basis to whether ve could pull 
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together materials that would take a student from vnere he might be 
to where he wanted to go on an individualized basis, About six 
months prior to the research stage, ve developed and installed the 
first learning lab. To date, we have 94 comprehensive learning Jabs 
and .aore than 150 mini-labs. The comprehensive libs include 
subjects from grade levels one through fourteen, and many labs are 
especially designed to meet the needs of thf adult basic student. 

Since the inception of the learning lab program here in 1964, it 
has expanded in other states, and probablv 20 additional states are 
using a similar type of system at the prisent time. They developed 
into Job Corps programs, Youth Center and HIND projects, public 
school projects, Neighborhood Youth Corps, correctional centers, 
colleges and university projects. 

In order for you to better understand what they are, let me 
try to define a learning lab and also the methodology employed, 

The learning lab can best be defined as an accumulation of commercially 
available programs and/or self instructional materials. These mater- 
ials are arranged by subject in a logical sequence in relationship 
to grade level and general interest. The instructional technique is 
dependent upon the methodology which it follows. 

As a teacher is a key person in a school or college, so the 
coordinator is the key person in the learning lab. The role of the 
traditional teacher must change when a new instructional technique 
is introduced. The coordinator serves as the facilitator in the 
learning process by bringing the student and subject together in a 
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one-to-one ratio and having the student interact on a one-to-or.e 
'atio with the subject to be learned. I think most of us will agree 
you really can T t teach anybody anything, but you can facilitate the 
learning process by bringing the student and the subject to be 
learned together and letting tlieu! interact. The coordinator should 
be highly trained in programmed and self -ins true tional materials, 
capable of making educations . decisions and of directing students 
through their assignments. The coordinator interviews, counsels, 
and seeks to establish and to develop an effective line of communica- 
tion with the student. 

Placement inventories are used to assess the student’s entering 
behavior in reading, math, and English. The reading inventory is 
always given, but the English and math are given later if the student 
is reading below 5th grade level. 

During the counseling process t he coordinator indirectly 
encourages the student to state roughly his objective. Probably 
this is the first time that the student has stated in his own 
words what he would like to be able to do. The learning lab 
instructional program is developed r?n a sound instructional basis 
that is divided into four parts; (1) first is the stated objectives, 
or what ve expect the student to be able to do at the end of instruc- 
tions; (2) second is our assessment of the student, to determine 
his entering behavior, (3) next ve provide instruction which is on- 
step, and meaningful to the student; and (4) last we evaluate his 
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progress. Any time you have an instructional program or system 
wif-h these four components, you have the basis for a good system. 

As I have mentioned, the coordinator is the key person. There 
is a very high correlation between the coordinator's ability and 
interest and the success of the learning laboratory program. If 
this person says the program is not going to work, you can be 
assured that it it, not going to work because he or she will not 
allow it to work. 

We have installed these learning labs in just about every 
corner and nook you can think of, purchased all materials at the 
state level, and took the material out to the lab as a gift to 
the institution. No matter how biased or prejudiced the president 
or administrative staff might be, or whether the project staff were 
stuck in a corner, the laboratories have done an excellent job 
regardless of such handicaps. The coordinator is the key person. 

During the counseling process, the coordinator helps the 
student to state verbally his objectives or what he would like 
to be able to do at the end of instruction. Sometimes tlu student’s 
desire is to learn to write his name. Not only do we help the 
student achieve this goal, but hopefully, higher and more complex 
tasks will be perceived and desired by the student. This, in 
many cases, is the beginning of an effective verbal contract 
between the student and the coordinator. The establi shment and 
maintenance of an effective line of communication and verbal contract 



186 - 




serves as a guarantee that the st ; 'ent will remain with us for a 
Longer period of time than he might otherwise do. This contract 
leads to greater motivation. 

Each student is given a diagnostic inventory in reading and 
math. These adult students have failed test after test, but they 
have never failed an inventory; so when you tell them you would like 
for them to take the placement inventory they relax. But, if you 
say ,r test", they freeze up. This is just to determine their level 
of frustration. From these assessments an individual curriculum 
is tailored to meet the needs of each student. 

This curriculum is developed for the student in relationship 
to his reading ability and stated objectives- Having evaluated a 
student by means of counseling and testing, the coordinator lists 
the programs that will take the student from where he is to where 
he wants to go. Here the student ivill quite often take part in 
selecting the material or program which he likes best, so you get the 
student involved in making decisions at an earLy date. If the 
student objects to a prescribed course, and since the student is always 
right, we let him start in a course of his choosing. He knows the 
door is open for him to coir.e back and to start over in a more realistic 
subject and at a level wherein he can achieve success. 

The student is thoroughly introduced to programmed instruction. 
This is most important because all of us have developed a pattern of 
behavior and ve function in this pattern of behavior. The adult 
student functioning at a very low level perceives of a learning 



situation as being in a classroom with one teacher and approximately 
thirty students. You have to re-structure his perception as Lo 
the learning process or as to his role in the learning process. 

If you fail to do this, the student will not adjust properly. So 
again the coordinator is the key. 

We know thei e are several ways to learn, such as by listening 
or reading, but the coordinator's main purpose here is to re-orient 
the student to individualized instruction, A v:orking relationship 
is established between the coordinator and the student, whereby the 
student knows the program will teach him on £ one-to-one ratio, 
but at the same time he is confident in the fact the coordinator 
is there to help him over any hurdle that he may confront from 
time to time. 

Students are placed at a level wherein they achieve immediate 
success. They are challenged by the more difficult material but 
are not bored by remedial skills. They are motivated as a result 
of their Success in the more difficult material. This is what I 
refer to as on-step instruction. The whole process of the learning 
lab is that you never ask a student to do something that he is not 
capable of doing. Initially, \<e want to place the student just 
prior to on-step so that he can achieve success, and then move up to 
his on-step level. 

This on-step can be defined as the maximum level of arousal in 
the learning process which can best be provided in the learning lab 
on an individual basis. The student sets his own best pace that 
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will allow at least an p *5% rate of comprehension. Once the adult 
student has been introduced to the program and understands what 
he is supposed to do, he mo^es at his own rc^c. There is no outside 
competition, and the frustration in most cases has been eliminated. 
If suudents are placed too low, they are allowed to skip concepts 
or sections until they are challenged. This is to say, they are 
kept on-step to maintain maximum motivation in learning. 

Since there are no formal classes, a realistic schedule is 
established for eacn student. This is important because most adult 
stc ents will do what is expected of them if they understand and are 
properly directed. The schedule is developed relevant to his family 
and work responsibilities. The student takes part and helps the 
coordinator decide on a definite time and number of hours per week. 
After a realistic schedule has been developed and established, the 
student is told that he will be expected to attend at a specified 
time. The schedule includes a minimum of two hours of instruction 
per subject per week. If the students core less than four hours 
per week, they will not achieve the sense of accomplishment to the 
point that they will sustain the initial motivation. 

In addition to the established schedule, the students are 
encouraged to work in the lab at any time, and as many times a 
week as they desire. This type of scheduling does not interfere 
with job and home responsibilities. Really, a lot of frustration 
adhering to a strict schedule has been elir.irUcd cnee you mb’ 
this type of program available to the students. Quite often, tho 
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coordinator will tell the student, ir If you can't come, call me ,T and 
when they do call that's great, because you know you have a good 
relationship with them. 

Chapter, unit, and post test are administered to be sure the 
students obtain 85% or higher on comprehension » If maximum progress 
is not being made, program changes are made. Each lab contains 
the standardized test kit watch includes the following; algebra, 
geometry, general math, trigonometry, reading, English, comprehension, 
and general science, government, history. 

The physical facilities that exist in various learning labs 
vary; however, the basic requirements will include r.uch items as 
a coordinator's desk, tables, chairs, a student cabinet assigned 
to each individual, file cabinets, and approximately 220 linear 
feet of shelving. Hopefully, there will be a room large enough 
to adequately accommodate about 30 students. The coordinator's 
desk should be located at some spot where he or she can control 
the traffic. Since the majority of students are enrolled in 
reading, the materials are displayed on a large table near the center 
of the room where students can get to them without creating a 
traffic problem. We have found that tablet arm chairs are not 
desirable at all because there is too much material for the student 
to manipulate and do adequate work. 

We have found that anytime you us^ a device without audio 
equipment, after the novelty wears off it is nothing more than 
manipulating the software. If you are using audio equipment, you 
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should use the audio and visual together. We use such items of 
equipment as the Language Master, tape recorder (reel and cassette), 
and filmstrip viewers. At one time, we used the single Super-8 
Concept projector, but found the films for this projector too 
expensive and not very interesting to the adult students. Presently, 
we are developing more of a multi-media system than we had before. 

My opinion is that the cassette recorder will really be a great 
innovation in the field of education. We have found that the adult 
students really like using the cassettes, or tapes. 

Objectives of learning laboratories pertain to programs at any 
level of comprehension. The labs are basically designed to help 
adults achieve the following goals: (1) to give basic instruction 

and literacy training from grades one through eight, and (2) to 
prepare students who request it for the General Educational 
Development test. About four years ago, ve tested the validity 
of the GED curriculum that had been developed. At that time, 
we had 600 students that came into one particular lab with an 
average reading level of eighth grade. In a period varying from 
twelve to eighteen months, each student completed his curriculum 
and had taken the GED test. Out of the 600 students, 588 passed 
the test the first time* I dare say that you could not get the 
same results with such a heterogeneous group with the same number 
of hours with any traditional classroom approach. 

During the spring quarter of 1969, there were 5,692 students 
actually enrolled in the 74 comprehensive labs with 3,015 or nearly 



53 percent below eighth grade. The average entrance level was seventh 
grade. Fifty percent* or 2*870* were male; 2,240* or 39 percent 
are non-white; 3,327, or 58 percenc were enrolled for high school 
preparation; 1,231, or 21 percent were in academic programs; 840, 
or 14 percent in occupational trades; and 294, or 5 percent in 
general interest courses. The total cost of comprehensive labs 
will vary anywhere from $4,000 to $15,000 depending on the number 
of students to be served and available funds. The learning labora- 
tory system is flexible. 

I would like to share with you some of the accomplishments 
of the learning lab thus far. It has raised non-readers to literacy 
level, and a few to high school graduation. This required initial 
and comprehensive instruction in reading, English* math* science* 
and social studies. It has provided the basic academic skills 
necessary for admittance into the curriculum and the mastery of 
vocational level training such as auto mechanics, brick masonry, 
and similar trades. It has provided academic skills and knowledge 
necessary for admittance into the mastery of technical and college 
level instruction. 

A survey was conducted several months ago to find out how 
the adults were doing who had gone through our learning labs and 
received their high school diplomas or CED certificates and had 
enrolled in two-year college programs. We wanted to compare their 
performance with the average high school graduates. \<?. found that 
80 percent of these students were in the top 20 percent of their 
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classes . 



I would liks to sum up for you some of the advantages of a 
learning lab. Every student in the learning lab achieves success 
as a result of the nature of programmed instruction and the 
methodology used. Furthermore, success contributed to higher 
motivation among the students. If the students come to you, 
even though they are motivated, they will not maintain the motiva- 
tion if they are not successful. 

I believe that success is the greatest factor in sustaining the 
motivation of the students. The instructional material used in 
the lab reduces student frustration, since each student sets his 
own pace based on his rate of comprehension. Personality conflicts 
are significantly reduced since the students interact with the program 
text and a coordinator is always present. The labs are designed to 
meet individual needs on a variety of bases; educational, social, 
and psychological. 

Scheduling Is flexible; for example, one fellow who had 
dropped out of high school in the ninth grade spent four years 
in the navy, and came back wanting a high school diploma. He 
came into the lab at 8 a.m. and stayed until 5 p .n. for five 
days per week. At the end of five months, he had completed 
his high school curriculum. After passing the GED test with a 
high score, he decided to go to college. The coordinator Informed 
him that he needed to come back to the lab and spend additional 
tine in reading, math, and science. He then spent six more months 
on these courses. He passed the SAT test with a score of 1200. 




This year, he is a junior in Electrical Engineering at Nor: 

Carolina State University. 

The nature of the learning lab instructional material compels 
a student to progress in a logical manner with minimum deviation 
from the program as outlined. Lab students nave a higher rate of 
comprehension of such materials than non-lab pupils. Students 
retain this information much longer than if they acquire it in a 
classroom situation. The mode of instruction in the lab is more 
valid and more comprehensive in addition to being subject to less 
bias. The program offered te the students in the lab has greater 
contiruity than its traditional counterpart in the same program. 

You will see the continuity built in once you see the curriculum. 

The materials used in labs are designed for adults, regardless of 
the grade level for which they are designed. The cost per student 
contact hour is less in this instructional system compared to the 
cost in a traditional classroom. 

In summary, if the student can't read the program, he is not 
placed in that particular one. If he needs math at the seventh grade 
level, and he is reading at the fourth grade level, you must have 
a program with seventh grade skills but written at the fourth grade 
level. You never ask a student to do something he is not capable 
of doing. If he is not successful in one program, ve switch him 
to another program. Programs are selected that are relevant to the 
student's needs, likes, and dislikes. In curriculum development, the 
student takes part in selecting materials that will take him from where 
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he is to his objective. Scheduling is flexible and students may be 
phased into a program at any time and phased out when their objectives 
have been achieved. In the learning laboratory our motto is "Learn- 
ing by the yard is hard, but learning by the inch is a cinch". 
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SYSTEMATIC USE OF PROGRAMMED INSTRUCTION' 
IN BASIC EDUCATION 

Grady M. Meredith 

Rehabilitation Research Foundation 
Draper Correctional Center 



Introduction 

The process iu which programmed instruction (PJ.) is currently 
used in the basic education program at Draper Correctional Center 
is the result of almost constant change. Revision of techniques, 
trials of new and different methods of approaches are continuing, 

From the beginning of PI to the present, Draper has been very much 
involved in its application to adult education and, at times, in the 
development of programmed material. 

In 1962, Dr. John McKee began an educational program at Draper 
with small grants from the Ford Foundation, the. Alabama Board of 
Corrections, and the Aaron Norman Fund. Later he received grants 
from the National Institute of Mental Health (NIMH) which allowed 
further research to be conducted and experience to be gained in using 
PI with institutionalized adults. More materials were being produced 
and techniques for using these materials were developing. Experiments 
were conducted to try to determine more effective ways of utilizing 
them. Out of chis experimentation evolved the beginnings of the 
method we now use. 

In 1964, the first of three Manpower Development and Training 
(MDT) Experimental and Demonstration (E&D) projects was funded at 
Draper. Basic, or remedial academic training (using PI), social-skill 
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training, and training in several trades were offered and rigidly 
scheduled. Trainees in this project were required to attend basic 
education classes two hours per day throughout the program regard- 
less of their grade level. 

During the three years which followed, we conducted additional 
E&D projects which provided further opportunities to develop and 
refine the procedures we now use. In our present project we are 
requiring of a trainee only that academic, training which io 
necessary for success in the trade he ha: selected. lie may elect 
to go beyond this minimum and work toward i certificate of high 
school equivalency or possibly to prepare for college. Many have 
successfully done this. 

T raine e Characteri st ics 

This bit of history brings us to the present and the current 
undertaking at Draper. U f e are now conducting our fourth project, 
which is funded under Section 251 of the Manpower Development and 
Training Act (MDTA) . 

The oft-described characteristics of the under-educated adult 
require no elaboration here. There are, however, two characteristics 
which must be noted because they are of paramount importance in 
planning and implementing training programs for this population. 

The first is their attitude toward education. Having experienced 
only failure in public schools, they, in a learning situation, 
are usually either apathetic and numbered, or are hostile, suspicious, 
aggressive, and resistive. They are apt to view any educational 
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activity as an insurmountable obstacle course, Prison inmates, the 
disadvantaged group with whom 1 have had most of my experience, may 
treat the educational program as a game of wits, with authoiity as 
the opponent. Though they will conform, it is usually only to 
secure earlier release or to avoid an undesirable work assignment. 

The second charac teri Stic is their diversity in levels of 
knowledge. One vocational class of 80 had grade placement scores 
ranging from third grade to second-year college. This uneven 
pattern is an individual as well as a group characteristic. Under- 
educated adults, similar to people on any level of education, do 
not usually have a total absence of knowledge, rather, they have 
knowledge gaps. If grade placement scores on subtests of standardized 
achievement tests are charted, there will be peaks and valleys, 
I mportance of Ind ividual ization 

These characteristics, poor attitudes and knowledge gaps, 
emphasize the importance of providing an educational program to 
meet the needs of the individual. His training program must be 
related to his own goals, and the re la t ionship must be clearly 
defined for him. (The first step may even be to help him to set 
realistic goals.) Thus, to attempt to teach him only whac he 
needs to know - one must concentrate on the valleys. And the 
valleys, which are his general deficiencies, must be carefully 
plumbed. It's like walking a tightrope. If we err in one direction 
we can attempt to teach material which the trainee has already 
mastered. If, on the other hand, we lean too far in the other 
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direction, we may preset, material for which the trainee is not 
prepared. It is desirable, therefore, to determine as precisely as 
possible what each trainee needs to learn. 

The disadvantaged trainee should not be frustrated, but he can 

be frustrated if he is asked to study material he already knows or 

material at a level for which he is not ready. 

\ 

These characteristics also mean that a conventional classroom 
approach had best be avoided. The conventional classroom generally 
recalls a series of failures to a man who has a deep-seated need for 
success experiences . Further, the conventional classroom seldom 
takes into account his unique pattern of knowledge and ignorance. 

This is a wordy way of saying that, at all costs, the disadvantaged 
trainee must be provided with a program which is designed specifically 
for him. 

Thn uiost effective tool we have found for use in our basic 
education program is PI. We have built our entire basic education 
program around this medium. Our efforts have been directed toward 
developing a systematic approach, utilizing this relatively new 
teaching technique. It has been successful in our program where other 
older approaches have failed. In our situation it is the only tool 
which offers the flexibility ve need. Any trainee can use PI 
materials if he is able to read - at the third- or fourth-grade level. 

Reasons for Using PI 

Some of the reasons we have adcDted this teaching technique are: 

1. PI materials are adaptable to almost any physical situation. 
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Trainees may work in specially designed study areas furnished 
with computers, or they may secure a paper and pencil program 
and work at their bunks. 

2. PI offers the only method we know which will allow up to 15 
trainees in one learning area simultaneously, each studying 

a different subject at a different level of difficulty. This 
one feature makes it worthwhile for our program. 

3. It. enables us to show an average grade gam of 1.4 grade 
levels in 208 hours oi instruction. This includes all 
levels of training. Some large companies quote fantastic 
grade g: ins in extremely short times with the use of their 
materials, such as 4.5 grade levels incr f ase in 55 hours of 
instruction. Put whan we learned what this meant, w T e 
discovered that these figures represented gains by trainees 
who were at approximately third-grade level at the outset 
of training; and this is where the fastest rate of learning 
occurs. They did not include those who began at fifth-grade 
level or above, where the rate of learning is usually slower. 

4. It assists us in preparing over 95 percent of our candidates 
for passing tie General Kdicat ional Development Test. (These 
trainees are selected on the basis of overall test scores, 
approximately eighth-grade level, upon entry into a six- 
month vocational and basic education training program.) 

5. PI allows complete freedom and flexibility in individualizing 
the learning process for each trainee. Ko can treat specific 
deficiencies with no waste of trainee tine. 
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6. It provides some intrinsic motivational characteristics 
which are sorely needed by adults an well as children. 
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One of the most outstanding of these is that the trainee 
is able to succeed - something which he probably has not 
had a great deal of success in doing previously. 

Characteristics of PI 

In addition to PI being extremely flexible, it offers advantages 
which are not found in the materials which have been traditionally 
used in the classroom. The following characteristics are advantages 
which distinguish PI from other materials: 

1. Pi's training objectives arc behavioral, that is, the trainee 
will be able to do something specific after completing a 
program. 

2. Material is presented to the trainee in a logical sequence 
in which concepts and/or behaviors are broken down into 
steps which are small enough to be easily learned. 

3. The trainee is required to respond actively to each step, 
that is, he is asked to supply information or perform 
skills immediately after they are introduced. If his 
responses do not require him to perform precisely the 
kind of behavior w T e are trying to teach him, they require 
that he approximate the behavior. When the response cannot 
be elicited in a real-life situation, such a situation is 
simulated . 



- 202 - 



204 



4. The trainee receives immediate feedback to each response-he 
knows at once that he is right or wrong. If he is wrong 

he is told why. In good PI he is almost always right. This 
characteristic explains the behavior-shaping power of PI. 

The knowledge t'o: a response is right in reinforcing to the 
learner. When he encounters similar situations in the 
future, he will be apt to make the same response. 

5. Most definitions of PI will state that it is self-pacing, 
that is, trainees will learn at their own rate. Programs 
designed for use on an individual basis have this character- 
istic, and it is an important one. Indeed, it is this 
characteristic which makes a system such as we describe 
work. There are, however, programs for use by snail 
groups of trainees whose skills and knowledge in a 
particular area are at about the same level. We use one 
such program at Draper to improve reading skills. 

PI, because of the characteristics just mentioned, is ideal for 
use in an individualized system, thus permitting the teacher to 
handle a variety of learning problems within a single class. For 
example, a trainee who needs to learn a specific basic skill, such 
as how to add whole numbers, might be given one progra , while t lie 
trainee who has completed all of his assigned work and is anxious to 
move on to the next step might be given a different program which 
could enrich his learning either in breadth or in depth. As you 
will note, PI materials can be used in a variety of other ways: 
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as a means of recalling previously learned subject matter, to supple- 
ment classroom study, or to enable the trainee who has missed classes 
for various reasons to do make-up work and so on. 

The Learning Sys tem 

We believe there are four basic, critical steps in the systematic 
use of PI materials to obtain the type of results in basic and reme- 
dial education mentioned earlier. In sequence, they are: (1) diag- 
nosis. (2) prescription, (3) management, and (4) evaluation. 

Diagnos is 

Background data must be collected. This should include age, 
marital status, socioeconomic background, employment, last grade 
attended or completed in public school, length of time since 
attendance in school, personal problems, and so on. / 

A test must be selected which can be used as a diagnostic 
instrument. We have used the California Achievement Tests and 
Metropolitan Achievement Tests. All levels of the selected tests 
should be available, as should alternate forms of aLl levels for 
post testing or continuing evaluation of the trainee r s progress. 

Other tests are available, some of which are designed for adults. 

The tests which are selected should render a grade placement score, 
not percentile and stanine scores; the latter nre of little use in 
diagnosing deficiencies when the result of the diagnosis is to be 
a prescribed study program. The primary reason for this is that 
PI materials which have been properly tested should be described in 
terms of the grade level for which they are suited. 
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Besides rendering grade placement scores, otLer features are 
desirable. Ease of administration and scoring can conserve staff 



time. Tests for which an item analysis is available are preferable; 
item analysis is indispensable in making a refined diagnosis. The 
test should provide some measure of reading ability, stated in terms 
of grade placement. One final but important point for consideration 
in selecting a test is cost. The expense of operating a comprehensive 
testing program csn be considerable and should be carefully considered 
in terms of the budget. 

Once a battery of tests has been selected, a further determination 
must be made regarding the proper level of the test to be administered 
to each trainee. We nave used the GRAY ORAL READING TEST as a rough 
indicator in selecting the most appropriate level of the standardized 
achievement test to be administered. If this is not possible, the 
first section of the standardized achievement test can be administered 
and the trainee observed. If it is obviously too difficult or too 
easy for him, he shculd be given another level of the test* We 
understand that some tests will soon be available which will have 
built into them some means of selecting the proper level for individ- 
ual trainees. 

An item analysis of test results gives the manager (we refer to 
the teacher or instructor as a ’'manager") the first general indication 
of a trainee’s weaknesses and strengths - the peaVs and valleys 
mentioned earlier. It would seem that from an item analysis, one 
should be able to prescribe PI which would t i eat the deficiency, 
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but this is not the case if we are to be specific in prescribing 
remediation. For example, the SAT scores might show a deficiency 
in computational skills - this tells us nothing. Wien the item 
analysis is applied, it might reveal a weakness in division of 
fractions. This is a bit more helpful, but still not enough. 

The trainee might know everything about dividing fractions except 
that he must invert before multiplying. Determining this require 
a more refined diagnosis. A pre- or post- test for a specific math 
course which covers fractions could be used to pinpoint the gap 
in his knowledge of fractions. 

The result of this painstaking procedure is that the trainee 
can be assigned a portion of a unit within a course which will 
correct the deficiency, rather than an entire math course. This 
results in saving time and efforr for the trainee along with other 
positive factors, such as allowing him tc progress at his own 
rate and not tying him to a group which might move too fast or 
too slow. Since the entire procedure is individualized, the 
possibility that a trainee may be criticized by his peers for lack 
of skill or knowledge is reduced. 

Pre s cripti on 

After analyzing the available data and weighing it in terms of 
the trainee's goals, we prepare a prescription schedule for each 
trainee. The schedule is a record of the courses to be assigned and 
the order in which the trainee will take them. To .repare it, the 
instructor must .onsider the information lie has gained during the 
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diagnostic process. This information will include achievement test 
results with overall and subtest scores and a measure of the 



level at which the trainee reads, the item analysis, any further 
diagnostic test results, and any information gained during interviews, 
including the trainee's goals and identifiable attitudes. 

The instructor, after having selected and ordered PI materials 
for his trainees, will be familiar with the materials in terms of 
behavioral objectives, grade level, reading level, appropriateness 
for age level* and format or method ox presentation. 

Keeping in mind the age and approximate grade level of the trainee 
plus the information and insight he has gained into the trainee's 
abilities and interests, he weighs course objectives against defi- 
ciencies and selects the course (ox courses - or possibly only a 
portion of a course) which the trainee will begin to study. At 
this point, the prescription is still tentative. It may have to 
be revised at intervals. 

Ma n agement of the Lea :ni ng Activities 

Before the trainee actually begins his studies, he should have 
a counseling interview with the manager. During the interview, 
the instructor will review and interpret the trainee's test scores 
and show him how they were used to prescribe the programmed material 
he will use. The explanation should be couched in terms of the 
trainee's goals. At this time the trainee's commitment to his 
prescribed course of study should be secured. The means of properly 
using programmed materials must be explained, as should testing 
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and grading procedures. Any questions the trainee may have should 
be dealt with. 

When the trainee begins to work with these new materials, he 
must be observed closely for the first few days. Any errors in 
prescribing should be corrected as soon as possible. Supervision 
must be maintained. Verbal feedback from the trainee, progress 
checks, and observations by the manager are all valuable tools 
which can. be used in involving the trainee in his own learning 
process. 

All of the aforementioned procedures require that the manager 
be thoroughly familiar with the program's content, grade level, 
and method of presentation, as well as with the performance, work 
habits, and specific def it iencies of the trainee if he is to success- 
fully manage a program of this kind. He must also be aware that 
trainee performance tan vary substantially Decause of factots such 
as the mode of presentation of the materials (a paper and pencil 
program as opposed on one which requires a machine), a trainee’s 
interest in the subject matter, and the relationship of the subject 
matter to the trainee’s academic or vocational goals. In addition, 
the manager must frequently search for reinforcers which can main- 
tain desirable learning behaviors. 

It has been demonstrated that productivity can be increased 
by utilizing contingency management procedures in the educational 
program. For example, a point value may fce assigned to each 
programmed course on the basis of its length and difficulty, and. 
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on the basis of available money, a monetary value oi the points may 
be established. L'sing these procedures, a student can earn money 
based on bis own productivity. He is not, however, paid for merely 
completing frames. He has to pass unit and/or final tests in a 
course in order to earn points. 

Such a plan need not be rigid in delivering reinforcement. 
Students may be paid once a week or they may be paid whenever they 
have accumulated a certain number of points; or payment for points 
earned in one course may be made contingent upon earning points 
in another course, / point system is an extremely flexible device 
which allows the manager to schedule reinforcement and vary t lie 
value of points and the contingencies of earning them as he sees 
the need. The learner should receive reinforcement only after 
he has performed specified tasks which meet the established 
criteria in tbs. educational system. 

Other motivating techniques, such as progress charts or graphs, 
may be used. The trainee's own academic goals, when discussed with 
him, are often highly motivating. Other goals which depend upon 
more adequate basic education, such as higher level vocational 
skill* 7 , may be used to motivate Lrainees, Many of our trainees 
are not aware of t lie most basic requirements for advancement. life 
goals, if they exist, arc free* eutly established in childhood through 
the influence n social contacts. The disadvantaged are not usually 
exposed to the infiueTces which cause individuals to give thought 
to predominant life motives. but these motives can be developed in 



educational training, particularly if progress in basic education 
can be related to personal advancement. 

Evaluation 

Any learning system must have provisions for evaluation. The 
most practical means of evaluating student or program progress is to 
adninister a test. To measure a student r s accomplishments in the 
basic education program, we administer a different form of the same 
standardized achievement test which was initially given. if Form AM 
of the Metropolitan Achievement Test (advanced) was administered 
when the trainee was admitted to the program, Form BM of the same 
test is administered when he leaves. By comparison of pre- and post- 
test results, we can ev \luate the trainee‘s progress (which we feel 
is also an excellent evaluation of our procedures), comparison of 
pre- and post-test item analysis should give an extremely accurate 
analysis of procedures. 

In order to evaluate any procedure, detailed records must be 
kept. All test results, including standardized achievement tests 
and tests on assigned courses, are recorded along with all pertinent 
dates, manager’s comments, and so on. By carefully recording all 
pertinent data, we are able to maintain an ongoing evaluation of 
our diagnostic and prescription processes. 

Other factors involved in evaluation of a program are frequently 
based on observation or discussion with trainees. A close look at 
the morale of trainees and staff, interest in learning (or the lack 
of it), and problems created by unacceptable behavior in the learning 
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area can provide supervisors with information which is invaluable 



in operating a program. 

In short, evaluation depends upon accurate and detailed record 
keeping (records which should be used — not just accumulated) , on- 
going observation of the trainee and the learning environment. 
Evaluation should be a contant procedure -- not something which 
is done when the course or project is completed. 



From our experiences we can now list a number of suggestions 
for training disadvantaged populations similcar to the one at 
Draper Correctional Center. However, these recommendations are 
merely guides to behavioral engineering and we still require some 
imaginative adaptations when employed in other settings. 

1. Use frequent and consistent reinforcements when the 
desired behavior is obtained, whether money, congratula- 
tions, or a pat on the back — anything to reinforce the 
desired behavior. 

2. Keep teps in learning small — to avoid any semblance of 
anticipated failure. 

3. Give frequent and positive feedback regarding performance. 

4. Carefully justify to the trainee the subject matter to be 
learned in a practical way - make his learning relevant 
to his future. 

3. Treat the disadvan taged as trainees, not students. The 
latter suggests to them that they are not adults. 



Re c o intend a t i o n s 
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6. Give tangible evidences of achievement , such as certificates 
of award, progress charts, p sting high test scores - 
remembering that it’s the educational system that is at 
fault when the learner fails. 

7. Provide a consistently positive atmosphere for the trainee. 

8. Id more than pay lip service to the concept of individualized 

instruction: practice it by using the latest developments 

in educational technology. 

Perhaps the disadvantaged are not much different from the rest 
of us when ^.t comes to training conditions and procedures. But two 
marked differences do stand out: their deficiencies and strengths 

are unevenly cast, requiring an individualized remedial program. 

Also, the disadvantaged generally have been deprived of that reinforcer 
we call success. We should give it to them offer.. 

Sources for P I 

; Just finding out vhat is available in PI material is a major 

| task. For this purpose, two resources appear to be indispensable: 

I "The Automated Education Handbook' 1 and the "Hendershot Catalog 1 '. Both 

references list programs by subject matter. Each also contains 
information about grade level, pi ice, and publisher. "The Automated 
Education Handbook" gives r. better description of the materials 
available. In addition, it contains essays and discussions on the 
theory and use of programmed instruction. The handbook is, however, 
an expensive volume; the caLalog may suffice strictly f or ordering 
purposes. To acquire programs to evaluate one peruses tlrjse volumes 
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and orders examination copies of programs which appear to cover 



the desired subject matter at the appropriate level an 
price which is within available means. 

No programmed instructional materials of system o 
tion is intrinsically valuable. It must be considered 
of target population, adaptability to curriculum, time , 
characteristics, measurable outcome, and budget. 
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DEVELOPING PROGRAMMED INSTRUCTION (P.I.) 
SYSTEMS AND MATERIALS 



Virginia Zachert 



Consultant, Medical College 
Augusta, Georgia 



Whether a program meets the needs that you have can only be 
determined if you try it out and see for yourself* You have to 
try out programs ("TOP 1 ') in your own setting. 

Don’t believe the propaganda that is attached to programs 
unless it is indicated that they were tried out on your type 
population. Programmed instruction should be "guaranteed learning 1 ' 
and if it is not, then you do not wart it. But when it is guaranteed 
to do one thing, and you want it to no something else, don’t be 
surprised when it does not work for you. 

Probably no technique has oeen more incorrectly used than 
programmed instruction because I say "it w T orks fine", you tvy it 
and it doesn’t work for you. But you are trying it out on one group 
of people and I have evaluated it on another group of people. Any 
type of teaching can work, and work fairly w T ell, in a situation where 
it meets the need. Most methods can not be used to meet different 
needs. They often f^il with some group. 

People keep saying that programmed instruction i* going to 
save a lot of time, a lot of effort. This is not so. If you 
have an efficient method of teaching, you will find that in 
programming it you will (1) not hurt the student, (2) not save 
any time, (3) not save any money. But if you have an ineffective 
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way of teaching you will find that this is probably the greatest way 
of moving from the 18th century lo the 10th century that has bean 
developed and it will move you there in one step. The reason: 
what you do in PI is unique; you l isten to the_ learne r l 

Programmed instruction Is an efficient and effective way of 
getting from the subject matter experts what they consider the 
learner needs to know. Most teachers are really not good at 
defining what the learner needs to learn. In my opinion, teachers 
have only two jobs. One is to define what should be learned and 
the other to see if it is learned. Only subject matter experts 
can define the parameters of what nee 3 to be learned as they know 
what "isn't to be learned". They have to have more knowledge than 
they are trying to communicate before being a good teacher. A 
teacher has to know more than he is going to teach so lie can 
determine what should be taught and what left cut. 

/ teacuer has to see that learning takes place. That is to 
evaluate. Many times . * i will find that if the parameters of what 
is to be learned are determined, you have only to write a complete 
final examination. Then anybody to whom you would give an "A" 
would know everything they need to know about the subject. Once 
you know what the objectives are, teaching is easy, or, at least, 
easier . 

In writing programmed material our biggest job in the medical 
profession is getting the professors to determine the boundaries. 
This is hard work.' 
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People will tell you that to properly analyze your job (look 
at what you want taught) requires at least 50 percent u£ your effort. 
In other words, if you are going to teach a college course, you 
should spend as m^ch time in writing your final examination as you 
spend during the entire quarter teaching that material! See why 
this is revolutionary? 

In an industry, the person who controls Lhe money controls 
the business. In learning, the person who controls the evaluation 
controls the learning. If you tell students ‘‘these questions are 
going to be on your final examination” you can be sure what they 
are going to study. You have determined their learning. 

If there is a logical sequence in what is to be. taught, 
the teacher has the responsibility of finding and teaching it. 

That is the reason the subject matter experts must analyze their 
material and find out w T hat part of it must be learned. What 
programmed instruction writers have done, and good teachers have 
always done, is to get at the schema. That is, get at the under- 
lying relationships of the material to be learned. And this is what 
a systems analyst tries to do; he tries to force the expert to 
look at his materiel a second time. What is it you really want 
taught? Programmed instruction is unique in getting at the 
"objective" (what you want taught) and devising a schema. This 
is wbat has been added to the old teacher -learner process. 

Everybody writes programs a litLle differently. And when 
anybody tries to teach you the way he d~es it, listen to him 
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and thank him because it is interesting, and then ignore it, You 
:\o n.jre want to copy somebody's meth d u£ writing programs than you 
want to copy Hemmingway’s style if you are going to write a novel 
You should write your own way when you write material just as you 
teach your own way. Your persona] to ich in teaching needs to come 
through. There are many things that you have to do to find out 
for yourself just what you want taught. I don’t care what method 
or technique you use — usually the sub je* t natter; the learners or 
your personal style will determine this. 

This week you have a tremendous group of people here. Every- 
one of them has come up with his solution to the problem, and each 
is going to show you his solutions to the problem. Tne first thing 
you do when you leave here is forget what they said as far as their 
solutions are concerned. The way they arrived at these solutions 
is what you should be interested in. How did they go about doing it? 
What underlay the preparation of material? What are learning processes 
Do not believe anybody when he tells you this is according to 
’’learning theory”. If we had a single learning theory that was 
adequate, teaching would be simple. But t-here are many learning 
theories and if you want' to set up a model for a doctorate in 
experimental psychology In the field of learning, you have to 
set up a minimum of five models to see which your controlled 
learning fits. We are babes in the woods as far as knowing about 
learning so I suggest you be an eclectic pragmatist: anything 

i*bat works is acceptable! You try it out and see if it works. 
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You must try out methods as well as techniques and definitely try 
out materials. 

Most of you are not going to be developing materials; most of 
you arc going tc be using them. You need to know whether they 
will work with your papulation, with your people. Try then, out 
and see. It doesn’t matter whether it is a video tape of a slide 
or a piece of paper, or whether ic T s a learning situation where 
they are working with three dimensional tools. Whatever works is 
what you want to use. 

To be sure you do it efficiently and effectively you must 
analyze your needs first. Spend half of your time determining 
what it is students need to know for this job. What are important 
things that you want done on this job. 

What is the underlying schema or what are the aspects that tie 
the whole thing together. We found that professors at the Medical 
College of Georgia are not interested in students* just learning 
facts. They want them to know how to use this information. Hov 
do you find out if a person can use facts? 

In most professions and in most trades there is evaluation to see 
if a person can do a job. How do you evaluate? In medicine the 
student is expected to selva a problem or to be able to work with 
a patient. But what really happens in a medical school? A student 
works up information on t. » patient; a res T aent physician is also 
getting information on this patient; case history, physical exam- 
ination, lab studies, etc. The medical student is expected to 
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decide what the problem really is and come up with the diagnosis, and 
what to do about it. hut the student knows good and well he is not 
going to be allowed to kill patients, so he considers much of this 
as merely an exercise. No matter what he does it won't make any 
difference. How can you put the student m a place where he can 
make errors and see the results of his errors? What would actually 
have happened to the patient if he did not get an adequate hiotory? 
What would have happened to the patient if he made a wrong diagnosis 
°i treated him in an inadequate fashion? Kow can you put him in 
a real situation to show him what is happening? You can't turn him 
loose on patients. But the student must be in a position to actually 
see that his error Is going to kill a patient. 

We prepared programmed problem solving cases and the student 
goes through these branching programs and is allowed to make errors 
and continue on through the branches. When he gets to the end it 
may say something like: "Unfortunately the kidney you took ont 

of this patient was the only kidney left; you did not do an adequate 
history an.. . iysical examination; go back and re-work this case." 

Another time when the student has gone through certain choices 
he may see, The patient lived, but unfortunately you butchered her. 
Her husband is seeing another physician and if he finds out how 
unnecessary the operation was you may be slapped with a mal-practice 
suit . " 

These rases are real to the student. I have watched medical 
students who have complete control of gathering data and material on 
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a patient (working to the best of their knowledge end with the 
best of intentions) find they have harm d and not helped the 
patient! These students will turn as white as a sheet when they 
see the error, they start rubbing their hands, and even get up 
and walk out of the room. They are in a much more real situation 
on paper than they have gotten anywhere in medical school. They 
vividly see that good intentions and the level of knowledge they 
have are not adequate for them to take good care of patients. 

Talking to a group of physicians in South Carolina last 
month, I asked them what was the primary reason for the recent 
errors they had made. And without exception, everyone of them 
said that he did not have time to get an adequate history on .he 
patient. Each one said he should have found out something mere 
about the patient and, if he had, he v T ould have acted differently. 
Their problems were not lack-of-knowiedge but lack-of- time and 
are not given to solution by Pi! 

How does one prepare problem cases? The material must be 
prepared by people who have actual clinical experience. The 
best care is one in which a competent person has made a mistake. 
The case is prepared so this mistake u T on 1 1 be made by others. 

Programmed instruction is a general approacn. M PI" has 
become a part of the language and it means many things to many 
people. It is like the elephant and the blind men, different 
ones of us have approached different parts of it. 
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In mev ..cal school the vocabulary is an important part in 
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teaching factual information. As students go onto each 
different service for two or more weeks, they have to calk to 
the professors in the pvotessors' special languages and, believe 
me, every specialty has its own languaga. How do you teach this 
quickly to a student who i - ming on the service, so he hears 
correctly and asks goof questions? Each professor states what he 
wants; then we give this information to a student who is paid to 
write a text. This text must teach so that he would be willing to 
accept the grade that his classmate would make on an examination by 
the professor. Strange as it may seem, the’ can do it once the 
area has been defined. 

At what level should a third year medical student know a 
given subject? The amount and depth have to be determined by 
the professor, for they determine what the students don't need 
to know as well as what they need to know. The professor sets 
the boundaries and gives us a series of exam questions. V, T e let 
the students write or re-write in consultation with the professor, 
these questions. Then the student learns the answers and teaches 
his classmates. k T e tell a student that he has two months to 
learn the material and write a program. It takes him one month 
to learn the material but after he has programmed it he must teach 
his buody in a few hours. 

Usually, we car. get professors lo write material, once. It's 
hard work, time consuming, and they are already working 70 hours a 




week, so ve have found that the best writers are our student learners. 
Students like it for they are learning while they are writing, and 
they are getting paid for it. Writing material is such hard work 
that most people will do one good program and quit^ 

Help in writing programs is available from books and workshops 
as vrell as from professionals in the National Society for Programmed 
Instruction. These are several with material on programmed instruction. 

lhe Ar: id Forces have been working programmed instruction since 
World Jar II. Since then the military has been putting several 
million dollars a year into programmed instruction research and 
developmen t . 

In the United States Air Force Air Training Command alone, 
a list is prepared every few months for the Commanding General 
indicating every training school for officers and enlisted 
persons and stating how much of the course is in self teaching 
foimat with an explanation why the rest of it isn't . The military 
now trains its own programmers and its own analysts. Materials are 
now available in many technical fields that work beautifully in 
self-paced self-teaching. 

I repeat that my advise to every person who is in the area of 
teaching, supervising, or managing ‘'earning programs is to "TUP" 

(Try Out Programs) . 

I think that ycu will find that programmed instructions, 
par ticular ily programmed texts are good when you have determined 
the need, ar.d they are prepared to r.ect the nerd thus giving you 
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efficient tools. But 1 must say that, in my opinion, you use programmed 
instruction and carefully programmed texts as a last resort. Any 
other way that can be used to teach is usually less painful and more 
enjoyable so if they learn as well, use other methods. If these 
do not work, you really have a NEED, remember T, programmed instruction 
is like a girdle". It's uncomfortable, it's miserable, it's a 
straight jacket; but at times nothing else will take its place' 
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MULTIMEDIA PROGRAMMING 



Shirley B. Bitterlich 
Vice President 
General Programmed Teaching 
Palo Alto, California 



The world of instructional media can be a world of confusion 
because there are so many different types from which to choose. 

It has been said that we use media simply because they are there, 
and unfortunately this is sometimes true. One solution is to 
employ a variety — the more the better — without regard to appropriate- 
ness or need Another is to back off from the multimedia approach 
to instructional design, relying heavily on the familiar spoken 
or wr'tten word rather than daring to venture into the \v 7 ide world 
of media. But these are non-solutions; each, in its own way, 
sidesteps the issue. There is no simple solution, but it is 
possible to find satisfactory answers to the question: What 

should I use to teach t his topic to these students under these 
conditions ? 

To start, let us consider some of the possibilities involving 
visuals. Not everything can be visualized, but for the most part 
there are few presentations that could not be made more effective 
by the inclusion of some form of visuals. The only purpose may be 
to i.ake a dull topic more interesting, but this reason enough. 
Carefully selected visuals will usually cause the student to 
attend more closely to the verbal content of what would otherwise 
be a routine and relatively ineffective presentation. Thus, it is 
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an appropriate use of the vi ual medi im and one that we tend to 



ove rlook 



There is, of course, much more to visualization than interest 
value. All too often we instruct verbally when it vouid be far 
better to use visuals as the primary source of instruction. For 
example, when we talk or write about a concrete object, we form 
a mental picture of it and then describe what we see in that 
picture. The trouble is that there is no guarantee that our 
mental picture will be transferred without change to the student. 

The mental picture he forms will probably be incomplete, inaccurate, 
or bjth. The more complex the object, the more the problem is compounded. 
For instance, how would you describe a spiral staircase to a student 
who had never seen one? At best it would be difficult, more likely it 
would be impossible, and most certainly it would be a waste of time 
to even attempt such a description. All that is needed is sh w him 
a picture and say, "This is a spiral staircase". And you could be 
sure that the next time he sees one there would be a better chance 
that he would recognize it than you had used the proverbial 
thousand words. In both this and the previous example, there would 
be a wide choice as to the type of visual to use. Photograph, line 
drawing, black and white, color — any of these would do, provided it 
furnished an accurate representation. 

In almost any type of vocational course, there is a considerable 
amount of discrimination t aining that precedes actual on-tue-jnb 
practice. Visuals are extremely useful in teaching such discriminations. 
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Here, however, the selection of the particular type of visual becomes 
more critical, and the characteristics of the target population and 
the constraints of the instructional setting, as well as the nature 
of the subject, may ent^r into the decision. For example, one of our 
company's recent projects involved teaching nurse's aides to discriminate 
between correct and incorrect hospital procedures. Photographs could 
not be used, primarily because there was too much extraneous detail 
(wrinkles in clothing and bed linens, shadows, superfluous objects 
such as t o 11 . j t articles, permanent wall fixtures, etc,). For 
instructional clarity, it was necessary to use original art, and 
developing the final art style required a long series of tryouts. 

Black and white line drawings v^re acceptable to the 1 .itracting 
0 rganizat ion— bu t not to the target population. For some reason 
that has still not been precisely determined, v:e had to use colored 
drawings , even though color itself was not a critical element of 
the basic discrimination. Line drawings, no matter how highly 
cued with labels, "failed” in developmental testing. 

In some instances it was necessary to show T progressive move- 
ments, for example, how T to bend down, pick up an object on th-*. 
f Inn r , and then return to an upright position without suffering 
body strain. We found that three body positions — all the w T ay dowrn, 
halfw 7 ay up, and standing upright — provided an approximation to 
actual movement that was close enough to preclude having to go to 
the intricacies of animation. 
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Verbal information was presented on audiotape in the form of a 
dialogue between an instructor and a student. This scriptwriting 
technique was largely determined by the characteristics of the 
target population. Even though the training wa^ provided in an 
individual setting, the aides simply did not feel at ease \ v f hen the 
instruction was aimed directly at^ them, which interfered with their 
learning. But when they watched somebody else being instructed, 
they no longer experienced any constrictive, personal involvement 
and entered wholeheartedly into the instructional situation. At 
the beginning of the sequence, the instructor would invite the 
viewer to participate by answering the questions he was going to 
ask of the student on the screen. Both the viewer and the student 
marked their answers on a response card, and then either the 
instructor or the student gave the confirmation. (As an additional 
self-confidence builcier for the viewer, the student did n j 
always get the correct answer!) Note that the hospital training 
program involved a combination of two media; audio and visual. 

Another course developed by our company resulted in a mix of 
several media, each one dictated by the nature of the subject matter, 
the characteristics of the: target population, or the conditions 
under which the instruction was to be provided. The project concerned 
the development of a self-instructional course to teach the soldering 
skills needed to qualify tor employment as an electrical assembler. 

Job performance involved not only numerous discriminations, but 
also a variety 0 f skills ranging from gross motor sequences to very 
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precise manipulations. Note that the instructional burden did not 
end when the trainee had acquired the preliminary discrimination 
training. In this case, there was jto^ on-the-job practice. Thus, 
the course had the added responsibility of teaching the actual 
performance skills, In addition, it had to be in the form of a 
complete do-it-yourself kit. It had co include all the tools and 
supplies that the trainee would need It had Lo include all of 
the instructional components, both software and hardware. And 
it had to include everything the course monitor would need to 
evaluate the trainee's performance. The monitor could not be 
expected to have any degree of expertise in this particular 
performance area; he would probably be as unskilled as the 
entering trainee. 

The course was first designed in skeleton form in the 
terms of behavioral objectives, and was then expanded to include 
the descriptive information required to teach the various kinds 
of discriminations and procedural steps that the trainee had to 
complete successfully. This, of course, also involved determining 
the order in which the steps should be taught. 

Two factors entered into the selection of audio for all verbal 
directions. First, the reading ability of the target population 
was very low, approximately fourth grade level. This factor, 
however, was less important than the second. Ke would have used 
aud^o regardless of the level of reading ability. This particular 
course involved performance from beginning to end. It w*as necessary 
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that the trainee be able to watch what he was doing at all times. 
’’Reading 1 ' his way through the course would interfere with his visual 
attention to his performance, but with audio instructions this 
problem was eliminated The speed of the audio p icsentat ion had to 
be carefully timed. Everything the trainee did — picking up a tool, 
using it, putting it down and selecting another — had to be taken 
into consideration. 

The second media input x^as visual , and numerous visuals were 
required throughout the course. The Primary purpose was to give 
the trainee a standard against which to compare his own performance, 
in other words, to provide the means for making the necessary discrimi- 
nations. The visuals were generally in the form of drawings so that 
the item could be enlarged and more emphasis could be placed on details. 
The illustrations were housed in a stand-up book directly in front of 
the trainee. He had merely to lift his eyes momentarily from his 
work, without moving his head, in order to make a visual comparison. 
There was no instructional advantage to be gained from projected 
visuals, nor was there any need for color. In fact, color reduced 
the effectiveness of the illustrations. (This was in direct opposi- 
tion to the findings in the hos ital •'raining programs, where color 
was equally unnecessary but had to be used to obtain effective 
visualization. ) 

There were some objectives that could not be achieved with 
still pictures because a time factor entered Into the sequential 
performance of the requisite procedures. This involved another 
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visual input--mot ion .picture* Note that we did not use color or 
projection because they were not needed, and we did use motion 
pictures because we could not do without them. In each case the 
selection of the appropriate medium was not made on the basis of 
an arbitrary decision. We did not select the v 1 ual media. Rather, 
they selected themselves in that they were determined by the performance 
requirements of the pertinent objectives, 

It proved impossible to clarify all of the visual features of 
soldered connections with two-dimensional drawings, which meant 
that still another medium was requir ed--the actual object. This 
provided the trainee with the necessary three-dimensional view, in 
addition to allowing him both to observe and to feel surface texture. 
Again, the selection of the appropriate medium was a function of 
the instructional need. 

In summary, the design of multimedia programs should follow 
the same basic approach used in the design of any other type of 
presentation. Specific media should be selected, not because they 
are fun to use nor because they happen to be there- Media, like 
procedures and techniques, should be chosen primarily on the basis 
of the performance requirements of the objectives, then modified 
as necessary to conform to the needs or limitations of the target 
population and instructional settln 
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Please make the following corrections on your copy: 



P. 3 - 

P. 16 - 

P. 20, 
P. 24, 



P. 24, 
P. 69, 
P. 75, 
P. 93, 
P. 95, 
P. 96, 
P. 105, 
P. 107, 
P. Ill, 
P. 128, 
P. 131, 
P. 132, 
P. 144, 
P. 144, 

Pc 150, 

P. 150, 

‘ P. 157, 

! P. 158, 

P. 160, 
P. 161, 
P. 162, 
P. 163, 
P. 166, 
P. 177, 
P. 179, 
P. 180, 



P. 203, 
P. 220, 




"Norbest Wcnier "--change to "Norbert Wiener 1 

"Norbest Weiner n --change to "Norbert Wiener 1 

line 4 - "accomplished" instead of "ac comlished" 

line 5 - change "images generated on paper and^ images" 
to ’’images generated on paper are images" 

line 9 - after learner insert "more" 

line 16 - "per hour ,T --not "per hours" 

line 4 - insert "3." before "Like McKee's work’ 

line 4 - insert "deviations" after "significant" 

line 22 - insert "who" between "people and "wanted' 

line 15 - "are able to" rather than "are chute" 

line 20 - insert "it" before "is" 

line 1 - "hierarchy" instead of "hierarch" 

line 22 - "ancient Sumer” rather than "Summer' 

line 15 - "vague" instead of "vauge" 

line 2 - "product" instead of "produce" 

line 16 - "corollary" instead of "collary" 

line 2 - capital.- for Educational Advancement Center 

line 6 - "a short test" instead of "short tests" 

line 9 and line 19 - "Suppes" not "Syppes" 

line 10 - colon after CAI instead of a semicolon 

line 22 - insert "the" before "Physics Project" 

line 14 - "relatively" instead of "relative" 

last line - "100,000" instead of "1000, 000" 

line 20 - insert after "possibilities" 

line 16 - change "mundance" to "mundane" 

line 22 - change "On the" to "One" 

line If - insert "The" before "Project plan" 

line 22 - change "Uka" to "Ute" 

line 24 - insert "in" after "considerations" 

line 10 - insert "(by Jacobs, P. I., Maier, M. H. and 
Published by Holt, Rinehart and Winston)." 

line 5 - change "in" to "is” 

line 22 - delete hyphen from "malpractice" 

line 6 - insert a period (.) after "need" 

line 18 - insert "Is" after "this" 





S to lurow, 



L. M. 



